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BACKGROUND OF THE INVENTION 



Field of the Invention 



The present invention relates to an acoustic transmission method and an 



apparatus for transmitting signals using sound waves as the transport medium. 
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This application is based on patent application Nos. Hei 11-329914, Hei 



11-356827. Hei 11-363811, Hei 11-366345, Hei 11-366346, 2000-136716, 2000-136717, 
2000-248800 filed in Japan, the contents of which are incorporated herein by reference. 

Description of the Related Art 



animals (audible sound signals) and associated information signals concurrently to some 
machine, the audible signal and the associated information signal are sent through separate 
channels. 

For example, audible signal and information signal are sent as electrical signal to 
20 modulate the carrier wave, and are sent to the receiver side through electromagnetic waves 
and other media such as cables, and are used after demodulation by the receiver side. In 
such methods, it is necessary to provide respective transceivers for the purpose of sending 
an audible sound signal separately from another signal different than the audible sound 
signal. 
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Conventionally, when transmitting the sound that can be heard by humans or 
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That is, a total of more than two apparatuses are necessary, and complexity in the 
overall apparatus configuration is unavoidable. Also, apart from this problem, although 
there are methods of signal transmission based on ultrasonic waves as carrier waves, 
because ultrasonic sounds cannot be heard by human ears, it is necessary to convert to an 
audible sound at the receiver side, and the method cannot be used for the above purpose. 

As a technology similar to the technologies described above for transmitting 
sound (audible sound signals) that can be heard by humans or animals simultaneously with 
associated information signals, is disclosed in a Japanese Unexamined Patent Application, 
First Publication, No. Hei 8-37511 "Interactive broadcasting system and receiving system", 
Publication date, 6 February, 1996 (Reference 1), for example. This method is based on 
simply superimposing signals according to the DTMF (Dual Tone Multiple Frequency) 
format on the audible sound signals. 

However, according to this method, sounds that are not needed to be heard and 
are not of any interest to humans can be heard clearly as background noise. Therefore, it is 
very disturbing, and possibilities exist of misunderstanding the intended meaning of the 
original sounds represented by the audible sound signals. 

Therefore, such methods cannot be said to be suitable as sound-based information 
transfer means. 

SUMMARY OF THE INVENTION 



Therefore, an object of the present invention is to provide means to easily 
transmit sounds (audible sound signals) that can be heard by humans and animals and to 
simultaneously transmit associated information signals. Also, the present invention has an 



object to provide various systems based on the information transmission means. 

According to the present invention, the object has been achieved in a method for 
transmitting acoustic signals comprising: a synthesizing step for synthesizing an audible 
sound signal and another signal different than the audible sound signal to generate a 
synthesized sound electrical signal; an acoustic signal outputting step for converting the 
synthesized sound electrical signal to an acoustic signal and outputting the acoustic sound 
externally; a transmitting step for transmitting the synthesized sound electrical signal; and 
an extracting step for extracting said another signal from the synthesized sound electrical 
signal that has been transmitted. 

In other words, in the present invention, the audible sound signal and another 
signal different than the audible sound signal are synthesized electrically, using a data 
hiding technique, for example, under a condition that the auditory presence of the signal 
cannot be detected by human ears, and this is emitted from a sound generation apparatus 
(speaker for example) to carry out signal output. 

In this case, synthesizing means may use existing data hiding technique for 
embedding ID information and the like in voice sound information. The data hiding 
technique is reported in "Special Issue, "Electronic Watermark" protects multimedia age, 
Nikkei Electronics, 24 February 1997, (no.683), pp.99-124 (Reference 2), "Article, Data 
hiding technique to support electronic watermark (part one)", Nikkei Electronics, 24 
February 1997, (no.683), pp.149-162 (Reference 3), "Article, Data hiding technique to 
support electronic watermark (part two)", Nikkei Electronics, 10 March 1997, (no.684), 
pp.153-168 (Reference 4). 

At the receiving side, the synthesized sound is collected by a microphone, and 
from a converted electrical signal, signal is extracted. Therefore, persons near the speaker, 
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unaware of the auditory presence of the signal, is able to listen to the voice sound. 

Using such a method, it is possible to easily transmit voice sound signals and 
another signal different than the voice sound signal using a simple apparatus. Especially, 
when transmitting signals from the sending side by voice sound that can transmit through 
5 air, the apparatus on the receiving side is only a voice sound generation means represented 
typically by a speaker, and the receiving side is a sound collection apparatus represented 
typically by a microphone, and therefore, radio transceiver or wired transceiver is not 
necessary so that an advantage is that the structure of the overall system is simple and very 
economical. 

10 Also, accordingly, once the synthesized sounds are recorded, sound reproduction 

apparatuses in all kinds of apparatuses, such as personal computer, karaoke player, radio, 
television, tape deck, video deck, MD player, CD player, DVD player, analogue record 
player, DAT deck, MP 3 player can be utilized directly as a sending apparatus. Further, the 
recorded synthetic sounds, when they are recorded on a sound recording medium, can be 

15 distributed. 

Also, recorded synthetic sounds are able to be transmitted directly as direct data 
through transmission networks such as the Internet, telephone networks and broadcasting 
networks. Also, synthetic sounds can easily accumulated, modified, processed, analyzed 
and stored. Also, using one medium called sound, two pieces of information can be sent 
20 simultaneously. 

Also, according to the present invention, applying such information transmission 
means, it becomes possible to provide means for transmitting acoustic information, such 
as music, and control signals to control motions and sound generation of a robot to match 
the voice sound information simultaneously and easily. 



Also, according to the present invention, applying such information transmission 
means, voice sounds broadcast by radio and the like and such information as traffic 
information or sightseeing information or commercial information that can be used in car 
navigation purposes simultaneously. Also, in order to realize this, without having to install 
FM multiplexing apparatus and the like at the broadcasting station side, information can 
be embedded directly in the voice signals themselves, so that the system can be 
constructed very economically. 

Also, according to the present invention, utilizing the information transmission 
means such as the one described above, voice sounds broadcast through a radio and the 
like and URL information from the site on the Internet can be simultaneously transmitted, 
and by using the URL, it becomes possible to rapidly access the site. The receiver side, 
accordingly, does not need to copy the URL or to take notes or to memorize. Also, without 
altering the facility of conventional radio, information via the Internet can be accumulated 
in a car navigation apparatus. Also, because the access is made easier, the sponsor can 
expect to have an increased number of access to its home page. Also, promotion that links 
commercial broadcasting and home page can be realized. 

Also, according to the present invention, utilizing information transmission 
means such as the one described above, by transmitting voice sounds obtainable from TV 
receiver to a robot, through TV broadcasting and the like, it enables to control robot 
operation and sound generation. Also, it enables to provide feedback from the viewer 
through the robot, interactive television broadcasting can be realized. 

Also, according to the present invention, utilizing information transmission 
means such as the one described above, using only voice transmission means, it enables to 
send signals to control the robot from personal computers and the like. Accordingly, the 
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robotic system can be simplified and cost lowered. Also, because there is no need for a 
dedicated line in the computer system for controlling the robot, an advantage is that, while 
using the robot, other devices such as scanner, terminal adapter, printer can simultaneously 
be used. 

5 Also, according to the present invention, utilizing information transmission 

means such as the one described above, it enables to broadcast data relating to coupons for 
sales promotion along with commercial broadcast, and to extract the coupon data at the 
receiver side. Then, it enables to accumulate the coupon data in personal computer and the 
like on the receiver side, sending the coupon data to a web server through the Internet and 

10 the like, checking the coupon data received at the web server side and accumulate the 

coupon data for each receiver. Then, it enables to award special business incentive, such as 
price discount on commercial goods, to the receiver according to accumulated coupon data. 
Accordingly, the receiver side can increase the enjoyment of actively listening to the 
broadcast commercial, while the sponsor can benefit from effective advertising and 

15 increased potential sales. 

BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1A and IB are diagrams to show a first embodiment of the present 
invention. 

20 Figure 2 is a diagram to show a method of utilizing a frequency band in the first 

embodiment. 

Figure 3 is a diagram to show an example of a synthesizing apparatus in the first 
embodiment. 

Figures 4A and 4B are diagrams to show the operations of the synthesizing 



apparatus in the first embodiment. 

Figure 5 is a diagram to show an example of the structure of an extraction 
apparatus in the first embodiment. 

Figure 6 is a diagram to show a second embodiment of the present invention. 

Figure 7 is a diagram to show a method of utilizing a frequency band in the 
second embodiment. 

Figure 8 is a diagram to show the structure of the synthesizing apparatus in the 
second embodiment. 

Figure 9A and 9B are diagrams to show the operations of the synthesizing 
apparatus in the second embodiment. 

Figure 10 is a diagram to show an example of the structure of an extraction 
apparatus in the second embodiment. 

Figure 11A and 11B are diagrams to show a third embodiment of the present 
invention. 

Figure 12 a diagram to the third embodiment of the present invention. 

Figure 13 is a diagram to show an example of the structure of a synthesizing 
apparatus in the third embodiment. 

Figure 14 is a diagram to show an example of the structure of a machine 
controlling section in the third embodiment. 

Figure 15 is a diagram to show an example of the structure of a synthesizing 
section in the third embodiment. 

Figure 16 is a diagram to show an example of the correlation of audio signals and 
operation signals and machine operation in the third embodiment. 

Figure 17 is a diagram to show an example of the correlation of audio signals and 



operation signals and machine operation in the third embodiment. 

Figure 18 is a diagram to show an example of the structure of a synthesizing 
section in the third embodiment. 

Figure 19 is a diagram to an example of the structure of a synthesizing section in 
the third embodiment. 

Figure 20 is a diagram to show the basic structure of a fourth embodiment of the 
present invention. 

Figure 21 is a diagram to an example of the structure of a synthesizing section in 
the fourth embodiment. 

Figure 22A and 22B are diagrams to show examples of the relationship of the 
input signal and switching operation in the fourth embodiment. 

Figure 23 is a diagram to show an example of the structure of an extraction 
section in the fourth embodiment. 

Figure 24 is a diagram to show a second example of the structure of the 
synthesizing apparatus in the fourth embodiment. 

Figure 25 is a diagram to show a second example of the structure of the 
extraction apparatus in the fourth embodiment. 

Figure 26A and 26B are diagrams to show the basic structure of a fourth 
embodiment of the present invention. 

Figure 27 is a diagram to show an example of the chronological relationship of 
music signals and dance operation codes in a fifth embodiment. 

Figure 28 is a diagram to show the attitude of a dance operation of a robot and the 
corresponding dance operation codes. 

Figure 29 is a diagram to show an example of the structure of an extraction 



section in the fifth embodiment. 

Figure 30 is a flowchart to show the processing steps in the synthesizing 
apparatus in the fifth embodiment. 

Figure 31 is a diagram to show an example of the structure on the receiver side. 

Figure 32A and 32B are block diagrams to show the basic structure in 
Embodiment 6. 

Figure 33 is a block diagram of an example of the structure in the automobile side 
in the Embodiment 6. 

Figures 34A and 34B are block diagrams of an example of the structure in a 
Embodiment 7 of the present invention. 

Figure 35 is a block diagram of another example of the embodiment in 
Embodiment 7. 

Figure 36 is a diagram to show an example in Embodiment 7. 

Figure 37 is a block diagram of a functional structure of a robot in Embodiment 

7. 

Figure 38 is a diagram to show an example of the correlation data for the 
operational command for the robot in Embodiment 7. 

Figure 39 is a diagram to show an example of the correlation data for the 
conversational command for the robot in Embodiment 7. 

Figure 40 is a diagram to show another example in Embodiment 7. 

Figure 41 is a block diagram to show a functional structure of a robot in 
Embodiment 7. 

Figure 42 is a block diagram to show an integrated system of communication and 
broadcasting in Embodiment 7. 
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Figure 43 is a diagram to show an example of the correlation data for the 
operational command for the robot in Embodiment 7. 

Figure 44 is a diagram to show an example of the correlation data for the 
conversational command for the robot in Embodiment 7. 
5 Figure 45 is a diagram of the overall structure of the robot in Embodiment 9 of 

the present invention. 

Figure 46 is a block diagram of the functions in a personal computer side of a 
robot in Embodiment 9. 

Figure 47 is a block diagram of the functions in the robot side of the robotic 
10 system in Embodiment 9. 

Figure 48 is a diagram of the details of the robot control section in Embodiment 

9. 

Figure 49 is a diagram to show a relationship of signals in the personal computer 
side in Embodiment 9. 

15 Figure 50 is a block diagram of the functions in a variation of the robotic system 

shown in Embodiment 9. 

Figure 51 is a flowchart of the synthesized sound electrical signal generation 
process in Embodiment 9. 

Figure 52 is a block diagram of the functions on the personal computer side in 
20 Embodiment 10. 

Figure 53 is a block diagram of the functions on the robot side of the robotic 

system. 

Figure 54 is a diagram to show the relationship of each signal on the personal 
computer side in Embodiment 10. 
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Figure 55 is a diagram of the overall structure of the robot in Embodiment 11 of 
the present invention. 

Figure 56 is a diagram of the overall structure of the robot in Embodiment 12 of 
the present invention in Embodiment 12. 

Figure 57 is a diagram to explain a method of manufacturing a CD containing 
synthesized sound electrical signals in Embodiment 12. 

Figure 58 is a block diagram of the function on the robot side in a robotic system 
in Embodiment 13. 

Figure 59 is a block diagram of the function on the robot side in Embodiment 13. 
Figure 60 is a block diagram of an example of broadcasting commercial 
messages. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The following embodiments do not restrict the interpretation of the claims 
relating to the present invention, and the combination of all the features explained in the 
embodiments is not always being indispensable means of solving the problem. 

In the following, preferred embodiment of the present invention will be explained 
in detail with reference to the drawings. 
Embodiment 1 

First, examples in Embodiment 1 will be explained. 

Figures 1A and IB are diagrams to show Embodiment 1 of the present invention, 
and numeric symbol 1A refers to the sending side and IB refers to the receiving side. In 
Figure 1A, numeric symbol 1 represents audible sound signal, 2 an insertion signal, which 
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is another signal different than the audible sound signal 1, and 3 a synthesized sound 
electrical signal. 

Although the following explanations relate to a case of using a digital signal for 
signal 2, but even if the signal 2 is an analogue signal, once the signal has been converted 
to digital information by processing the signal through an A/D converter, it is obvious that 
it can be handled in the same manner as digital signals. 

Also, 4 represents a synthesizing apparatus, 5 an amplifier (in Figure 1A, it is 
recited as AMP), 6 a speaker, 7 a synthesized sound, 8 a person. In Figure IB, numeric 
symbol 9 represents a microphone, 10 an extraction apparatus, and other numeric symbols 
are the same as those in Figure 1A. 

An outline of the flow of the signal in Figures 1A, IB will be explained. First, on 
the sending side, audible sound signal 1 and signal 2 to be transmitted other than the 
audible sound signal 1 are electrically synthesized using, for example, a data hiding 
technique, under a condition such that the auditory presence of signal 2 cannot be detected 
by the person, and, after passing through the amplifier 5, a synthesized sound 7 is emitted 
from the speaker 6 into the air space. 

A person who is nearby can hear this synthesized sound by own ears. On the 
other hand, on the receiving side, the emitted sound is collected by the microphone 9, and, 
after passing through the amplifier 5, is converted to the synthesized sound electrical 
signal. Then, the signal 2 is extracted from the synthesized sound electrical signal by the 
extraction apparatus 10. 

Figure 2 is a diagram to show a method of utilizing the frequency band 
(band-elimination method) in Embodiment 1. Figure 3 is a diagram to show an example of 
the structure of the synthesizing apparatus in Figure 1A, and numeric symbols 1-3 are the 
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same as those in Figure 1A, 20 represents the synchronizing signal, 21 a switching circuit, 
22B a band-elimination filter 0, 22A a band-elimination filter 1 (band-elimination filter is 
abbreviated as BEF in the diagram), 23 a machine signal-switching signal converter, and 
24 a synthesizer. 

Figures 4A, 4B are diagrams to explain the operation of the synthesizing 
apparatus shown in Figure 3. Figure 4A shows the properties of the band-elimination filter, 
and numeric symbol 25 show the properties of the band-elimination filter 0 and those of 
the band-elimination filter 1, respectively. Also, Figure 4B shows the relationship of the 
waveform of the synthesized sound electrical signal and machine operation signal (signal 
2) and the changeover switch 21. 

Figure 5 is a diagram to show an example of the structure of the extraction 
apparatus shown in Figure IB, and numeric symbols 2, 3, 10 are the same as those in 
Figure IB, 20 represents a synchronizing signal, 27 a frequency analyzer (abbreviated as 
FET in the diagram), 28 an energy detector, and 29 a machine signal converter. 

Embodiment 1 will be explained in the following with reference to the diagrams. 
The synthesizer 4, as shown in Figure 3, is comprised by a band-elimination filters 22A, 
22B to eliminate a certain band from signal 1 depending on the value of the binary signal 
2, and when sending a code "1", the sound energy is eliminated in a particular central 
frequency A by the band-elimination filter 22A (BEF1). 

Similarly, when sending a code "0", the sound energy is eliminated in a particular 
central frequency B that is different than the central frequency A by the band-elimination 
filter 22B (BEF0). Humans are virtually unable to distinguish an audible sound uniquely 
lacking a specific frequency only from the original sound in their normal auditory state in 
daily living space. 
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Suppose that, even when such existence is able to be detected physically, humans 
are not able to meaningfully recognize its significance. Transmission of signal 2 is carried 
out as follows. 

(Step 1) the machine-signal-switching-signal converter drives the switching 
circuit 21, in accordance with the polarity of signal 2, while switching the 
band-elimination filter for signal 1 according to "1" and "0" to generate a sound signal 
from which a specific superposition has been eliminated from signal 1. 

(Step 2) after synthesizing the filtered sound signal in the synthesizing apparatus 
24, it is amplified in the amplifier 5 and is emitted into the air space through the speaker 6. 

Decoding at the receiver side is carried out by receiving the sound signal emitted 
into the air space in the microphone 9, and after amplifying in the amplifier 5, signal 2 is 
extracted in the extraction apparatus 10. In the extraction apparatus 10, the synthesized 
signal 3 input is analyzed in the frequency analyzer 27, the energy is detected in the 
energy detector 28, and the missing frequency component is extracted, and this is 
converted in the machine signal converter 29 into machine signals "1" or "0" so as to 
output as signal 2. 

Decoding at the receiver side can also be realized by another configuration 
different than the extraction apparatus 10. That is, the decoder is comprised by two 
band-pass filters, and by making their respective center frequencies equal to those of the 
band-elimination filters 22A, 22B of the synthesizing apparatus 4, the signal 2 can be 
decoded on the basis of the magnitude of the output from each band-pass filter. 

Specific receiving steps for signal 2 can be summarized as follows. 

(Step 1) detect a sound signal propagating through the air space by microphone; 

(Step 2) amplify the signal detected by the microphone; 
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(Step 3) obtain an output signal from a band-pass filter by passing the amplified 
signal therethrough; 

(Step 4) extract signal 2 by carrying out decoding according to comparison of the 
output signal with an appropriate threshold value. 

5 

Embodiment 2 

Figure 6 shows Embodiment 2 of the present invention, which shows both the 
sending side and the receiving side. In the diagram, numeric symbol 1 represents an 
audible sound signal; 2 a signal different than the audible sound signal; 3 a synthesized 

10 sound electrical signal; 4b a synthesizing apparatus; 6 a speaker; 7 a synthesized sound; 8 
a person; 10 an extraction apparatus; 11 a sending apparatus; 12 a signal transmission 
path; and 13 a receiving apparatus. 

An outline of the flow of the signal in Figure 6 will be explained. A feature of the 
present embodiment is that, as illustrated in the diagram by a dotted line, there is a means 

15 for transmitting (signal transmission path 12) the synthesized sound 7 from the sending 
side to the receiving side directly without propagating through the air space. In this case, 
an advantage is that, because acoustic noise in the air space is not mixed in the sending 
signal, reliability of signal transmission is increased. Also, at the same time, the 
synthesized sound is emitted into the air space and acts directly on the ear of a person. 

20 Figure 7 shows a method of frequency-band utilization in Embodiment 2. In this 

case, transmission method based on sine wave is adopted. The synthesizing apparatus 4b 
in Figure 6 is an apparatus to convert a binary signal into an acoustic electrical signal, and 
is comprised by an oscillator, so that, when sending a "1" code, a high frequency C having 
a specific frequency is generated, and similarly, when sending a "0" code, a high 
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frequency D having a specific frequency different than the high frequency C is sent. 

Humans are virtually unable to distinguish an audible sound uniquely lacking a 
specific frequency from the original sound in daily living air space in the normal auditory 
state. Even if such an existence can be detected physically, humans are not able to 
5 meaningfully recognize the significance of signal 2. 

Figure 8 is a diagram to show an example of the synthesizing apparatus in Figure 
6, and the numeric symbols 1-3 are the same as those in Figure 1A, and 4b represents a 
synthesizing apparatus; 20 a synchronizing signal; 21 a switching circuit; 23 a machine 
signal -switching signal converter; 24 a multiplexer; 31A an oscillator 0; and 31B an 
10 oscillator 1. 

Figures 9A, 9B are diagrams to explain the operation of the multiplexer shown in 
Figure 6. Figure 9A shows the frequency properties of the oscillator, and numeric symbol 
32 refer to the frequency properties of an oscillator 31 A, 33 to those of an oscillator 3 IB. 
Figure 9B shows a relationship of the waveform of the synthesized sound electrical signal 
15 and the machine signal (signal 2) in relation to switching by switching circuit 21. 

Figure 10 is a diagram to show an example of the structure of the extraction 
apparatus shown in Figure 6, and numeric symbols 2, 3, 10 are the same as those in Figure 
IB, and 27 represents a frequency analyzer (abbreviated as FFT in the diagram); 28 an 
energy detector; 29 a machine signal converter. Embodiment 2 will be explained below 
20 with reference to these diagrams. 

The synthesizing apparatus 4b as shown in Figure 8 is provided with oscillators 
31A, 31B for impressing different frequencies on signal 1, depending on the value of the 
binary signal 2, so that when sending a "1" code, a high frequency C having a specific 
frequency is generated, and similarly, when sending a "0" code, a high frequency D having 
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another specific frequency is sent. 

Sending of signals is performed according to the following steps. 

(Step 1) operate oscillators 31A, 31B for generating sine waves of specific 
frequencies (high frequency C, D) to correspond with "1" and "2" for signal 2; 

(Step 2) the machine signal-switching signal converter 23 operates the switching 
circuit 21 according to the polarity of signal 2 so as to add the selected specific sine wave 
and signal 1 in the synthesizer 24; 

(Step 3) the signal obtained in step 2 above is amplified in the amplifier (not 
shown), and the signal is emitted into the air space through the speaker 6 and, concurrently, 
the sending apparatus sends the signal to the transmission path 12. 

In the receiving side, the signal is received by way of the transmission path 12. 
The extraction apparatus 10 decodes the synthesized sound electrical signal again into a 
code signal. At this time, the extraction apparatus 10 analyzes the synthesized signal 3 
input therein in the frequency analyzer 27, detects its energy in the energy detector 28, and 
the impressed frequency component is extracted, and this is converted into machine 
signals "1" and "0" in the machine signal-switching signal converter, and is output as 
signal 2. 

Decoding at the receiver side can also be realized by another configuration 
different than the extraction apparatus 10. That is, the decoder is comprised by two 
band-pass filters, and by making their respective center frequencies equal to those of the 
oscillators 31A, 31B of the synthesizing apparatus 4b, the signal 2 can be decoded on the 
basis of the magnitude of the frequency components contained in signal 1. 

Specific receiving steps for signal 2 can be summarized as follows. 

(Step 1) receive an electrically synthesize sound signal propagating through the 
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signal transmission path 12; 

(Step 2) detect the specific frequency component in the received electrically 
synthesize sound signal; 

(Step 3) discriminate and output its polarity (binary value) according to the 
5 frequency component of the detected signal; 

(Step 4) extract signal 2 by carrying out decoding according to comparison of the 
output signal with an appropriate threshold value. 

Accordingly, two examples of the structure of sending side and four examples of 
the structure of receiving side are illustrated in Embodiments 1 and 2, but the data hiding 
10 methods are not limited to these mentioned, and similar acoustic transmission methods 
may be realized by using various other methods of data hiding algorithm. 

In the above explanations, specific examples of audible sound signal 1 include 
natural language audio sounds, machine synthesized sounds, musical tones, warning 
sounds, natural sounds in the natural world, animal sounds and noises. Also, signal 2 
15 represents data that are expressed in digital or analogue forms, and specific examples 
include natural language sounds, music, warning sounds, noises, MIDI (musical 
instrument digital interface) data, acoustic signals such as MP3, multimedia signals such 
as text, image and video signals, and sensory signals such as touch, smell and force. 
Also, the media for transmitting synthesized sound electrical signals in 
20 Embodiment 2 include, specifically, sound waves, ultrasonic waves, electrical waves, and 
electro-magnetic waves such as infrared, visible and ultra-violet radiations. Also, sending 
and receiving means may consider using broadcasting such as television and radio, CATV 
(cable television), police radio, public phones, portable phones, PHS (Personal 
Handy-phone System), Internet and LAN (Local Area Network). 
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Embodiment 3 

Embodiment 3 will be explained in the following. In Embodiment 3, a voice 
response machine that can respond to audio control signals is realized by applying the 
5 present invention to machine control. 

Media for enabling wireless remote control, without connecting a machine to be 
controlled and a control apparatus directly by means of conductive wires, such as 
electrical waves, infrared radiation, light and sound waves, have long been known and 
used. Of these, a representative method using sound waves in the audible frequency band 
10 is a based on commands comprised by synthesized sound signals that can be processed 
readily by machines. 

Such methods based on commands by synthesized sound signals that can be 
easily understood by machines have the advantage that recognition rate is higher than a 
method based on natural language, but the commands are difficult to be understood 
15 directly by humans. 

For this reason, in a communication air space in which machines and humans 
coexist, a method using two information channels has been adopted when mutual 
understanding of each other's intentions is necessary. 

That is, the natural language sounds are sent to humans by way of a speaker, and 
20 separately but concurrently, signals to control the machine corresponding to the 

information are sent to the machine by means such as electrical waves. In this case, it is 
necessary to have sending and receiving apparatuses for both natural language sounds and 
control signals by electrical waves and the like, resulting in a problem that the overall 
structure for the apparatus becomes complex. 
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Also, this method is basically powerless under situations such as underwater and 
hospitals where electrical waves cannot be used. However, as explained below, 
Embodiment 3 of the present invention provides means for readily communicating mutual 
intentions in the communication air space shared by machines and humans. 

Figure 11 A shows the configuration on the sending side in Embodiment 3, and 
Figure 11B shows, similarly, that on the receiving side. Numeric symbols 1-8 in Figure 
11A are the same as those in Figure 1A. Also, in Figure 11B, numeric symbol 211 
represents a machine control section, 212 a machine operation command signal, 213 a 
machine, and other numeric symbols are the same as those in IB. 

Features of the signal flow in Embodiment 3 are as follows. The signal 2 
extracted in Figure 11B is input in the machine control section 211, and is converted in the 
machine control section 211 into a machine operation command signal 212 to actually 
control the operation of the machine 213. The machine 213 operates in accordance with 
the contents of the machine operation command signal 212. 

The structures for the synthesizing apparatus in Figure 11A and the extraction 
apparatus Figure 11B can be used in Embodiments 1 and 2, for example. 

Figure 12 is a diagram for using a transmission path instead of extracting a signal 
for machine control based on the synthesized sound 7 that is propagated through the air 
space. As shown in Figure 12, the electrical signal output by the synthesizing apparatus 4 
is transmitted by way of the transmission path 224, and is input in the amplifier 5 in the 
receiving side. Other operations are the same as those in Figure 11A and 11B. 

Figure 13 is a diagram to show an example of the structure of the synthesizing 
apparatus shown in Figure 11 A. In this example, character strings in the natural language 
is used as signal 2, and numeric symbols 1-4 are the same as those in Figure 11A. 



Also, the numeric symbol 214 represents a machine signal/sound correlation 
section, 215 a voice data read section, 216 a voice data ROM, 217 an A/D conversion 
section, 218 an ASCII character code table, 219 an ASCII code converter, 221 a ROM 
addressing signal, 222 PCM (Pulse Code Modulation) voice data, and 223 an ASCII code 
5 signal. The voice data ROM 216 stores PCM voice data such as "GO", "STOP", "RIGHT", 
"LEFT" and others. 

In Figure 13, signal 2 is a character string of a natural language. For example, 
suppose that a signal meaning "GO" is input as signal 2. The ASCII code converter 219 
outputs an ASCII code 223 to correspond with "GO" by referencing the ASCII code 

10 character code table 218. 

This ASCII code 223 is input in the machine operation signal/voice correlation 
section 214, and the machine operation signal/voice correlation section 214 outputs a 
ROM addressing signal 221. The ROM addressing signal 221 is a data showing the 
leading address (of "GO" in this case) of the corresponding PCM voice data in the ROM 

15 voice data 222. 

The voice data read section 215 receives this and reads the PCM voice data "GO" 
from the ROM voice data 216 and outputs the data. The PCM voice data "GO" is 
converted to analogue data in the D/A conversion section 217, and becomes an audible 
sound signal 1. The audible sound signal 1 and the ASCII code signal 223 are input in the 

20 synthesizing apparatus 4 to be synthesized, and a synthesized sound electrical signal 3 is 
output. 

Figure 14 is a diagram to show an example of the structure of the machine control 
section, and the numeric symbols 2, 3, 10 and 211-213 are the same as those in Figure 11, 
and 225 represents an operation signal/machine operation command signal table. In Figure 
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14, the synthesized sound signal 3 collected by the microphone and amplified is input in 
the extraction apparatus 10, and signal 2 (operation signal) is extracted by the extraction 
apparatus 10. 

The signal 2 in this case is a signal comprised by a natural language character 
5 string corresponding to "GO". This signal is input in the machine control section 211, and 
the machine control section 211 outputs a machine operation command signal 212 
corresponding to the character string "GO" by referencing the operation signal/machine 
operation command table 225. The machine 213 operates in accordance with the machine 
operation command signal. 
10 Figure 15 is a diagram to show another example the synthesizing apparatus, and 

shows a case of using a binary signal for signal 2. The numeric symbols in the diagram are 
the same as those shown in Figure 13 explained earlier. 

In Figure 15, signal 2 is a binary signal, and is to be used directly as a machine 
operation signal. In this case, it is assumed, for example, that a machine operation signal 
15 to correspond with "COOL" has been input. The machine operation signal 2 is input in the 
machine operation signal/voice correlation section 214, and the machine operation 
signal/voice correlation section 214 outputs a ROM addressing signal 221. 

The ROM addressing signal 221 is a data showing the leading address (of 
"COOL" in this case) of the corresponding PCM voice data in the ROM voice data 222. 
20 The voice data read section 215 receives this and reads the PCM voice data "COOL" from 
the ROM voice data 216 and outputs the data. 

The PCM voice data "COOL" is converted to analogue data in the D/A 
conversion section 217, and becomes an audible sound signal 1. The audible sound signal 
1 and signal 2 (machine operation signal) are input in the synthesizing apparatus 4 to be 
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synthesized, and a synthesized sound electrical signal 3 is output. 

The operation at the receiving side is the same as that explained earlier based on 
Figure 14. 

As explained above, in Embodiment 3, the voice signal 1 and the correspond-ing 
machine operation signal (signal 2) are first synthesized at the sending side into one 
synthesized sound electrical signal by using the data hiding technique, and is emitted into 
the air space as a synthetic sound through the speaker by way of the amplifier. 

The meaning of the sound signal can be readily understood by a person who is in 
the vicinity by listening with own ears. In the meantime, at the receiver side, the broadcast 
sound is collected by the microphone, and is converted to a synthesized sound electrical 
signal through the amplifier. From this synthesized sound electrical signal, the machine 
operation signal (signal 2) is extracted by the extraction apparatus. 

Next, in the machine controller, the input machine operation control signal 2 is 
interpreted, and a corresponding machine control value is generated. The machine 
performs a specific operation according to the command value. 

Figure 16 is a diagram to explain a first example of the correlation of the voice 
signal and the operation signal and the machine operation, and shows a case of controlling 
a motion machine, such as a forklift, for example. The diagram in the column for the 
movement pattern in Figure 16 shows a top view of the motion machine, and the hatched 
portion represents a vehicle body and the four wheels attached to the vehicle body. 

The correlation of the voice signal and machine operation signal in controlling 
the motion machine is as shown in the columns in Figure 16. For example, if it is desired 
to move the machine forward, "GO" is pronounced as the voice signal 1. And, as the 
machine operation signal to correspond with this command, a natural language character 
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string "G, O" or, when using binary signals, ("0", "1") may be set. 

In this case, the natural language character string can be expressed as in the 
example of configuration of the synthesizing apparatus shown in Figure 13, alphabetical 
ASCII codes (8-bit information) corresponding to the natural language character string. In 
5 this way, one character in the ASCII code can be expressed by 8-bits. For example, "GO" 
becomes a 16-bit code consisting of "01000111" and "01001111". 

The synthesized sound is emitted from the speaker and is collected by the 
microphone at the receiver side. Concurrently, a person in the vicinity hears the machine 
sound "GO", and can readily understand the fact that a command has been issued from the 
10 sending apparatus to the motion machine to move the machine forward. At the receiving 
side, the following operations are performed. 

That is, the operation signal (signal 2) is extracted from the synthesized sound 
electrical signal received by the extraction apparatus 10. In the case of the above example, 
either the bit strings "01000111" and "01001111" expressing the character string "GO" or a 
15 binary signal ("0", "1") shown in Figure 16 is extracted. The meaning of the operation 
signal (signal 2) thus extracted is interpreted by the machine control section. 

Then, the operations signal is converted by the machine control section 211 into 
machine operation command signal to rotate the drive wheels of the motion machine in 
the forward direction as well as to set (correspond with "GO") the steering wheel in the 
20 straight forward direction, and is sent to the motion machine. The result is that the target 
machine moves forward. 

Figure 17 is a diagram to show a second example to correlate the voice signal 
with operation signal of the machine operation, and corresponds to a case of controlling an 
air conditioner. The correlation of the voice signal and the machine operation signal in 
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controlling the motion machine is as shown in the columns in Figure 17. 

In Figure 17, if it is desired to cool using the air conditioner, for example, 
"COOL" is pronounced as the voice signal 1. And, as the machine operation signal to 
correspond with this command, a natural language character string "C, O, O, L" or, when 
using binary signals, ("1", "0") may be set. 

The synthesized sound is emitted from the speaker and is collected by the 
microphone at the receiver side. Concurrently, a person in the vicinity hears the machine 
sound "COOL", and can readily understand the fact that a command has been issued from 
the sending apparatus to the motion machine to start cooling action of the air conditioner. 
At the receiving side, the following operations are performed. 

That is, the operation signal (signal 2) is extracted from the synthesized sound 
electrical signal received by the extraction apparatus 10. The meaning of the operation 
signal thus extracted is interpreted by the machine control section. In this example, it is 
converted to the machine operation command signal to operate the cooling apparatus in 
the air conditioner, and is sent to the air conditioner. The result is that the air conditioner 
begins cooling operation. 

Here, when the structure shown in Figure 12 is to be adopted, transmission of the 
machine operation signal is carried out directly using the transmission path 224 without 
propagating through the air space between the speaker and the microphone. The 
transmission path may utilize a signal transmission path for normal analogue and digital 
signals or an optical link. 

Figure 18 is a diagram to show still another example of the structure of the 
synthesizing apparatus. In the diagram, numeric symbol 1 represents an audible sound 
signal, 2 a signal, 3 a synthesized sound electrical signal, 4 a synthesizing apparatus, 226 a 
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voice recognition section, 227 a character string/operation signal conversion section, and 
228 a natural language character string. 

In the example shown in Figure 18, the features are that the audible sound signal 
1 is a natural voice sound input and that the signal 2 is a signal resulting from voice 
recognition of the input natural voice sound. 

The natural voice input sound is recognized in the voice recognition section 226, 
and is output as a natural language character string 228. This natural language character 
string 228 is input in the character string/operation signal conversion section 227 and is 
converted to the machine operation signal (signal 2). Then, the natural voice sound 
(audible sound signal 1) and the machine operation signal (signal 2) are synthesized in the 
synthesizing apparatus 4, and a synthesized sound electrical signal 3 is output. 

Figure 19 is a diagram to show still another example of the synthesizing 
apparatus. In this diagram, numeric symbol 218 represents an ASCII code table, 219 an 
ASCII code converter, and other numeric symbols are the same as those in Figure 18. 

In Figure 19, the natural language character string 228 output from the voice 
recognition section 226 is input in the ASCII code converter 219. The ASCII code 
converter 219 converts the natural language character string 228 to an ASCII code by 
referencing the ASCII code table 218. 

The signal comprised by the ASCII code is input as signal 2 in the synthesizing 
apparatus 4. The synthesizing apparatus 4 synthesizes this signal with the audible sound 
signal 1 (natural voice sound) to generate a synthesized sound electrical signal 3, which is 
output. Other structures and operations related to Figures 18, 19 are the same as those 
explained earlier, and explanations are omitted. 

Here, in the explanation for Embodiment 3, the example related to the case of 
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converting the natural language character string to the ASCII code, but this is not limited 
to the ASCII code, and it is obvious that other character codes can also be used. 

Embodiment 4 

5 In Embodiment 4, the purpose is to provide information transmission means that 

can be operated reliably without being affected by noise, even when the external noise is 
extremely high, so as not to cause errors in the transmitted information to trigger 
erroneous operation of the machine. 

Figure 20 shows the basic structure used in Embodiment 4. 

10 In Figure 20, numeric symbol 1 represents an audible sound signal, 1-1-1-3 are 

channels, 2 a signal, 3 a synthesized sound electrical signal, 304-1-304-3 amplifiers, 
305-1-305-3 speakers, 306 is a synthesizing apparatus, 306-1-306-3 synthesizing sections 
for respective channels, 307 a receiving apparatus, 308 an extraction apparatus, 309 a 
machine, 310 a transmission path, and 311 a person. 

15 In Figure 20, each audible sound signal in each of the channels 1-1-1-3 is 

synthesized with signal 2 in the respective synthesizing sections 306-1-306-3 of the 
synthesizing apparatus 306 to produce respective synthesized sound electrical signals 3. 
The synthesized sound electrical signal 3 is amplified in the amplifiers 304-1-304-3 and is 
output from the speakers 305-1-305-3 as acoustic sound. The person 311 is able to hear 

20 the sound. 

Further, the synthesized sound electrical signal 3 is sent to the signal transmission 
path 310. The receiving apparatus 307 receives the synthesized sound electrical signal 3 
and extracts signal 2 therefrom, and operates the machine 309 according to the signal 2. 

In present embodiment, as described earlier, the synthesized sound electrical 
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signal 3 is amplified in the amplifiers 304-1-304-3 and is output from the speakers 
305-1-305-3 as an acoustic sound, but pre-amplification audible sound signals in 
respective channels may be amplified and output from the speakers. 

However, as in present embodiment, by outputting after amplifying the 
synthesized sound electrical signal 3 in the amplifiers 304-1-304-3, the acoustic output of 
the synthesized sound electrical signal may be received in a microphone and the like, and 
it is possible to extract signal 2 from the output of the microphone, and therefore, the 
range of applicability is increased. 

Figure 21 is a diagram to show a first structure of the synthesizing apparatus. In 
the diagram, 1-1-1-5 represent channels, 3 represents a synthesized sound electrical signal, 
3-1-3-5 synthesized sound electrical signals in respective channels, 312-1-312-5 low-pass 
filters (abbreviated as LPF in the diagram), 313-1-313-5 changeover switches 
(abbreviated as S1-S5 in the diagram), and 314 a signal 2/switch operation signal 
converter. 

The signal 2/switch operation signal converter 314 in Figure 21, upon receiving a 
signal 2, converts this signal 2 to a switch operation signal, and this is given to the 
changeover switches 313-1-313-5 (S1-S5). The changeover switches 313-1-313-5 
receiving the switch operation signal perform switching operations according to 
pre-determined settings. 

Figure 22A, 22B show a relationship of signal 2 to the operations of the 
changeover switches S1-S5. In this example, as shown in Figure 22A, when sending "0" 
using the synthesized sound electrical signal, the changeover switches SI, S3, S5 are 
connected to the lowpass filter side, and when sending "1" using the synthesized sound 
electrical signal, the changeover swatches S2, S4 are connected to the lowpass filter side. 
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Figure 22B shows the relationship of the signal 2 in this condition to the operation of the 
changeover switches. 

Figure 23 is a diagram to show an example of the structure of the synthesizing 
apparatus. In Figure 23, numeric symbol 3 represents a synthesized sound electrical signal, 
5 3-1-3-5 represent synthesized sound electrical signals in each channel, 315-1-315-5 
highpass filters (abbreviated as HPF in the diagram), 316-1-316-5 energy detectors, 
317-1-317-5 threshold processing section, and 318 a majority logic section. As 
explained earlier, on the synthesizing apparatus side shown in Figure 21, depending on the 
value "0" or "1" of signal 2, control is exercised whether to output an audible sound signal 
10 by passing it through the lowpass filter or to output as it is. Thus, by passing the 
synthesized sound electrical signal through the highpass filter and analyzing the 
frequencies, it is possible to discriminate whether the synthesized sound electrical signal 
has been put through the lowpass filter on the synthesizing apparatus side or has been 
output as it is. 

15 In other words, when a frequency component higher than a specific frequency is 

contained in a signal, this signal has been output from the synthesizing apparatus side as it 
is without putting it through the lowpass filter. And, because it has been decided whether 
to pass or not pass through the lowpass filter for individual channels, depending on 
whether the value of signal 2 is "0" or "1", it is possible to discriminate whether signal 2 is 

20 "0" or "1" from the above discrimination result. 

Here, the cutoff frequencies of the highpass filters 315-1-315-5 may be set at 
approximately the same values as those of the lowpass filters 312-1-312-5 on the 
synthesizing apparatus side. 

The operation will be further explained with reference to Figure 23. The 
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synthesized sound electrical signals 3-1-3-5 in individual channels are put through 
respective highpass filters 315-1-315-5 and are input in the energy detectors 316-1-316-5. 

In the energy detectors 316-1-316-5, frequency components are analyzed and the 
results are determined in the threshold processing section 317-1-317-5. The output from 
the threshold processing sections 317-1-317-5 are input in the majority logic section 318, 
and the majority logic section 318 performs majority decision for each output of the 
threshold processing sections 317-1-317-5, and after determining whether signal 2 is "0" 
or "1", the results are output. 

In present embodiment, because the majority discrimination is performed on the 
signals embedded in a plurality of channels, the results are less prone to be affected by the 
noise, so that signal 2 can be transmitted more reliably, and their polarities can be 
determined more accurately. 

Figure 24 is a diagram to show a second example of the synthesizing apparatus. 
In Figure 24, numeric symbols 321-1, 321-2 represent audible sound signals, 3 represents 
a synthesized sound electrical signal, 322-1, 322-2 lowpass filters (abbreviated as LPF in 
the diagram), 323-1, 323-2 highpass filters (abbreviated as HPF in the diagram), 324, 
326-1, 326-2 are mixers (abbreviated as MIX in the diagram), and 327 an amplifier 
control signal generation section. 

In the example shown in Figure 24, the audible sound signals 321-1, 321-2, 
respectively, represent a left channel signal and a right channel signal of a stereo sound. In 
this diagram, the left channel signal 321-1 that passed through the highpass filter 223-1 
and the right channel signal 321-2 that passed through the highpass filter 323-2 are 
synthesized in the mixer 324, and are further input in the amplifiers 325-1, 325-2. 

The output from the amplifiers 325-1 is synthesized by the mixer 326-1 with the 
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left channel signal 321-1 that passed through the lowpass filter 322-1, and becomes a left 
E signal. On the other hand, the output from the amplifiers 325-2 is synthesized by the 
mixer 326-2 with the right channel signal 321-2 that passed through the lowpass filter 
322-2, and becomes a right E signal. Then, a synthesized sound electrical signal 3 is 
formed by the left E signal and the right E signal. 

The amplifier control signal generation section 327, depending on the value "0" 
or "1" of signal 2, controls the gain of the amplifiers 325-1, 325-2. For example, when 
signal 2 is "0", the gain on the amplifier 325-1 side is controlled so as to be 20 dB lower 
than normal. On the other hand, when signal 2 is "1", the gain on the amplifier 325-2 side 
is controlled so as to 20 dB lower than normal. 

Figure 25 is a diagram to show a second example of the structure of the 
synthesizing apparatus, and this synthesizing apparatus extracts signal 2 from the 
synthesized sound electrical signal output from the synthesizing apparatus shown in 
Figure 24. In Figure 25, numeric symbol 228-1, 228-2 represent highpass filters, 229 
represents a comparator. The cutoff frequencies of the highpass filters 228-1-228-2 may 
be set at approximately the same values as those of the highpass filters 223-1, 223-2 on the 
synthesizing apparatus side. 

The output from the highpass filters 228-1, 228-2 are compared in the comparator 
229, and are determined as "0" or "1" based on the magnitude, and are output as signal 2. 

As described earlier, in the synthesizing apparatus side, it is controlled so that, 
when signal 2 is "0", the gain of the amplifier 225-1 is lower than normal by 20 dB, and 
when signal 2 is "1", the gain of the amplifier 225-2 is lower than normal by 20 dB. 

Therefore, if the output from the highpass filter 228-1 in Figure 25 is higher than 
the output from the highpass filter 228-2, then signal 2 is "1", and conversely, if the output 



32 



from the highpass filter 228-2 is higher than the output of the highpass filter 228-1, then, 
signal 2 is "0". The output signal 2 (either "0" or "1") is given to the control section of the 
machine as a command signal, and is converted in the control section to a machine 
operation control signal to operate the machine. 
5 As explained above, in present embodiment, the sending side embeds signals in a 

plurality of independent channels, and the receiving side extracts embedded signals by 
comparing the signals in the plurality of channels, and therefore, binary signals can be 
transmitted at a higher stability and precision. 

10 Embodiment 5 

In Embodiment 5, the present invention is applied to a method of controlling the 
operation of a robot. 

For example, there have been toys robots that can dance to music. Specific 
examples include a toy robot called "dancing flower" which has an artificial flower or a 
15 doll that responds to a music and displays swing motion. 

This toy is constructed so that music and the like is collected by a microphone 
housed inside the artificial flower or doll, and is converted to electrical signals so that the 
artificial flower or the doll are made to sway by driving an actuator inside the dancing 
flower in accordance with the amplitude of the signals. For this reason, such robotic toys 
20 can only perform repetitive simple motion according to the sound pressure levels without 
relating to melody or rhythm of the music. 

To elevate the dancing motion of a robot to an artistic level, it is necessary to 
control its motion in such a way to be congruent with the musical texture represented by 
melody and rhythm of the music, that is, to choreograph the dancing motion. 
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In present embodiment, as described below, by embedding motion signals in the 
dance music for the robot, detailed movements of the robot can be controlled using a 
simple control structure without the need for information transmission means for 
customized motion signal. 
5 Figures 26A, 26B show the fundamental structure of the present invention. Figure 

26A shows the sending side and Figure 26B shows the receiving side. 

In Figure 26A, numeric symbol 401 represents a tone signal and 402 represents a 
dance operation signal, and other numeric symbols are the same as those shown in Figure 
1A. Also, in Figure 26B, numeric symbol 411 represents a robot control section, 412 an 
10 actuator command signal, 413 a human-shaped robot, 414 an operational mechanism of 
the robot, and other numeric symbols are the same as those shown in Figure IB. 

In the structural diagram of the receiving side shown in Figure 26B, although the 
microphone 9, amplifier 5, extraction apparatus 10 and robot control section 411 are 
drawn separately from the human-shaped robot 13, it is obvious that such constituting 
15 sections on the receiving side may be housed inside the body of the human-shaped robot 
13, or they may be housed inside a platform (not shown) that the human-shaped robot 13 
is mounted. 

Next, the overall flow of the signal will be explained. The sending side in Figure 
26A is constructed by replacing the audible sound signal 1 in Figure 1A with the musical 
20 signal 40 and signal 2 in Figure 1A with the dance operation signal 2. Therefore, to the 
stage of extracting the dance operation signal 402 using the synthesizing apparatus 10 in 
Figure 26B, it is the same as those in Figures 1A and IB. 

Next, the dance operation signal 402 is input in the robot control section 411 to 
generate internally an actuator command signal 412 (or command value) for the robot, and 



34 



the human-shaped robot 413 operates in accordance with the command signal or the 
command value. Person in the vicinity can observe the manner of robot motion with the 
peripheral vision while listening to the dance music to enjoy the dance performance of the 
robot. 

Next, a method of correlating the music with the movement will be explained. 
This is achieved by defining the relative chronological relationship of the tone signal 401 
to the dance operation signal 402. As an example, a case to be explained relates to the 
human-shaped robot shown in Figure 26B performing a dance operation routine using a 
music expressed by the musical score shown in Figure 27 to produce a dance routine by 
combining four dance operation patterns shown in Figure 28. 

Because the rhythm of this music is four beats, four dance steps are to be defined 
per one bar. Designating the dance operation code corresponding to the j-th beat of the i-th 
bar by Cij, the flow of the dance operation code of this music, that is, the code sequence 
can be expressed as, starting from the first bar: Cn C :2 Q 3 C 14 C 2X C 22 C 23 C 24 •-. 

Next, the dance stance of the robot, that is, as an example of the dancing pattern, 
four stances A, B, C, D shown in Figure 28 may be considered. That is, A is a stance with 
both arms up, B is a stance with both arms down, C is a stance with the right hand up and 
the left hand down, and D is a stance with the left hand up and the right hand down. 

In the following, to simplify the explanation, an example of the motion during the 
second bar will be explained. For example, if it is desired to dance in the order of D, C, B, 
A, the dance operation codes should be arranged as follows. 

C 2i =D, C 22 =D, C 23 =B, C 24 =A (1) 

This dance operation code, as the dance operation code 402, is synthesized with 
the tone signal 401 in the synthesizing apparatus 4. For this purpose, any of the 
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synthesizing methods explained in the forgoing embodiments can be used. 

At the receiving side, dance operation codes that are the same as equation (1) are 
extracted by the extraction apparatus 10, and the robot control section 411 receives these 
codes successively, and sends actuator command values to correspond to the code to the 
5 human-shaped robot 413. 

Here, when the tempo of the music is relatively fast, due to delay in robot control 
operation or in the transmission delay, it may appear sometimes that the robot motion lags 
the music. In such a case, the composite sounds should be generated so that the overall 
code sequence of the dance operation is increased to match the tempo of the music. By so 
10 doing, at the receiver side, at a certain point in time of music flow, the intended 

operational pattern code corresponding to that certain point in time has already been 
transmitted to the control section of the robot, so that the dance routine may be totally 
synchronized to the music without causing the problem of time delay. 

Figure 29 is a diagram to show an example of the structure of the synthesizing 
15 apparatus 4. In Figure 29, numeric symbol 401 represents a tone signal (in this case, 

music), 402 a dance operation signal (in this case, dance operation code), 3 a synthesized 
sound; 415 a musical operation compiling section, and 416 a code embedding section. 

In Figure 29, music 1 and dance operation code 2 are compiled in the music 
operation compiling section 415, and are synthesized in the code embedding section 416, 
20 and are output as a synthesized sound electrical signal 3. In the following, the operation of 
the synthesizing apparatus will be explained using a case of 4/4 beat. 

Designating N as the total bar number, mi(t) as an i-th bar tone signal, Pij(t) as an 
i-th bar j-th beat tone signal, and representing the time-series by symbols [;], [{}] , the 
original tone signal waveform Morg can be expressed as follows: 
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Morg (t) = {mi(t); m2(t); •-; mN(t)} (2) 

where mi(t) = {Pii(t); Pi2(t); Pi3(t); Pi4(t)}. 

And, the vector of the dance operation signal can be expressed as follows: 
Cdance = (Ci, Ci, •-, Cn) (3) 

where G = (Cn, Cn, Cn, Cu) 

Further, the tone signal waveform Mhyper synthesized on the basis of the tone 

signal waveform Morg and the vectors Cdance of the dance operation signal can be 

expressed as in equation (4) below: 

Mhyper(t) = {m hyper (i)(t) m hyper (N)(t) } (4) 

The synthesized signal waveform of Cij to Pij is expressed using "h" as equation 

(5): 

Pij %z Cij 

The term [m hyper (i)(t)] in equation (4) can be expressed as equation (6): 

m hyper (l)(t) = {Pll Cli; Pl2 %l Cl2J Pl3 X. Cl3; Pl4 Cl4> (6) 

The generation method of equation (5) is realized by the code embedding section 
416 shown in Figure 29. This generation method uses items explained in the foregoing 
example. 

The processing steps for obtaining the tone signal waveform Mhyper(t) shown in 
equation (4) from the Morg(t) shown in equation (2) and the dance operation signal 
waveform Cdance shown in equation (3) are as shown in the flowchart in Figure 30. The 
designations (S-l)~(S-9) in the diagram represent processing steps, and the steps S-l-S-9 
are referenced in the following. 

(step S-l), first, i is set to the initial value 1. 

(step S-2), next, j is set to the initial value 1. 
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(step S-3), a synthesized signal waveform of Cij to Pij is generated. 

(step S-4), then, it is examined whether j has reached 4. 

(step S-5), if j is less than 4, "1" is added to j and return to (step S-3). 

(step S-6), next, it is examined whether i has reached N. 

(step S-7), if i is less than N, "1" is added to i and return to (step S-2). 

(step S-8), Mhyper(t) is output, and the process is finished. 

Or, in step S-8, instead of outputting collected Mhyper(t), it is possible to output 
mhy P er(t) for every bar, and by so doing, signal embedding and signal sending can be 
performed in real time. 

Figure 31 is a diagram to show an example of the structure at the receiving side in 
the present embodiment. The flow of signals along the numeric symbols 7, 9, 5, 3, 10, 402 
in Figure 31 is the same as that explained in Figure 26B. 

Also, in Figure 31, 419 represents a dance operation code/joint angle conversion 
section, 402 a music tempo obtaining section, 422-1 a 61 target value signal, 422-2 a 02 
target value signal, 423-1 a rotation angle servo mechanism for motor 1, 423-2 a rotation 
angle servo mechanism for motor 2, 424-1 a motor 1, 424-2 a motor 2; 425-1 and 425-2 
motor drive current, 413 a human-shaped robot, 414 an operation section for the robot, 
426-1 a right arm angle (91) of the human-shaped robot 413, 426-2 a left arm angle (02) 
of the human-shaped robot 413. 

In Figure 31, the synthesized sound from the sending side is collected by the 
microphone 9, amplified in the amplifier 5, and its output, which is the synthesized sound 
electrical signal 3, is input in the extraction apparatus 10. And, the synthesized sound 
electrical signal 3 is also input in the musical tempo obtaining section 420. The extraction 
apparatus 10 extracts a dance operation code 402 from the synthesized sound electrical 
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signal 3 and outputs the dance operation code 402. 

The musical tempo obtaining section 420 generates a music tempo signal 421 
from the synthesized sound electrical signal 3 and outputs the music tempo signal 421. 
The dance operation code/joint angle conversion section 419 generates a 01 target value 
5 signal 422-1 and a 02 target value signal 422-2, and outputs the signals synchronously 
with the music tempo signal 421. 

The angle 61 is the right arm angle 426-1 of the human-shaped robot 413, and the 
angle 62 is the left arm angle 426-2 of the human-shaped robot 413. The rotation angle 
servo mechanism 423-1 of motor 1 receives the target value signal 422-1 of angle 01 and 
10 controls rotation angle of motor 1 (424-1). The rotation angle servo mechanism 423-2 for 
motor 2 receives the target value signal 422-2 of angle 62 and controls rotation angle for 
motor 2 (424-2). 

The operation section 414 is a mechanical operational section comprised mainly 
by the rotation angle servo mechanism 423-1 for motor 1, the rotation angle servo 

15 mechanism 423-2 for motor 2, motor 1 (424-1) and motor 2 (424-2). 

Summarizing, the present embodiment related to a case of sending a synthesized 
sound from the sending side and receiving in the microphone on the receiving side, and 
extracting the operation signal for the human-shaped robot from the synthesized sound 
electrical signal, but it is obvious that the synthesized sound electrical signal may be sent 

20 from the sending side to the receiving side through a wired transmission path, and the 
receiving side then extracts the operation signal from the synthesized sound electrical 
signal to operate the human-shaped robot. 

Embodiment 6 
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In Embodiment 6, road information and other information necessary for driving 
are provided to a navigation apparatus installed in an automobile. 

Drivers of automobiles would like to obtain, while driving, various fresh external 
information such as traffic information, weather forecast, sightseeing information and 
commercial information reliably, safely, simply and quickly at low cost. In the past, 
information for drivers has been provided through normal radio and television 
broadcasting. Also, in recent years in Japan, information providing services based on 
so-called "visible radio" through FM multiplexed text broadcasting, VICS information and 
car navigation system and others have also been realized. 

Of such information obtaining means described above, radio broadcasting enables 
information to be received readily inside the vehicle using a low cost facility so that it will 
continue to be used in the future. However, the driver is able only to listen to the content 
of radio broadcasting, and because the driver is not able to take notes during driving, it is 
difficult to retain records of important information. 

Therefore, present embodiment provides an economical means for providing 
voice information to car drivers through radio broadcast waves and data other than voice 
data concurrently. 

Figures 32A, 32B are block diagrams of an example of the basic structure of the 
present embodiment, and Figure 32A shows the sending side and Figure 32B shows the 
receiving side. 

In Figure 32A, numeric symbol 501 represents original sending sound (audible 
sound signal), 502 represents information for car navigation and the like different than 
original sending sound 501, 3 a synthesized sound electrical signal, 4 a synthesizing 
apparatus, 505 a sending apparatus, and 506 a sending antennae. 
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In Figure 32B, numeric symbol 507 represents a receiving apparatus, 508 a 
receiving antennae, 509 an amplifier (abbreviated as AMP in the diagram), 10 an 
extraction apparatus, 511 a speaker, 512 a synthesized sound, 513 a person (driver), and 
514 a car navigation apparatus, and other numeric symbols are the same as those in Figure 
32A. 

An outline of the flow of the signal in Figures 32A, 32B will be explained. First, 
on the sending side, the original sending sound 501 and the information 502 to be 
transmitted are electrically synthesized using data hiding technique, for example, under a 
condition such that the auditory presence of information 502 cannot be sensed by the 
person, and a radio signal modulated by the synthesized sound electrical signal 3 is 
emitted from the sending apparatus 505 through the antennae 650 into the air space. 
At the receiving side (automobile), the emitted wave is captured by the receiving antennae 
508, demodulated in the receiving apparatus 507 to extract the synthesized sound 
electrical signal 3. This synthesized sound electrical signal 3 is amplified in the amplifier 
509 and is output from the speaker 511 as a voice sound (synthesized sound) 512. The 
driver 513 is able to obtain voice information according to the synthesized sound 512. 

On the other hand, the synthesized sound electrical signal 3, which is the detected 
wave by the receiving apparatus 507, is input in the extraction apparatus 10, and 
information 502 is extracted from the synthesized sound electrical signal 3 by the 
extraction apparatus 10. The extracted information 502 is input in the car navigation 
apparatus 514 to perform desired operations such as display and the like such as 
displaying information on the display section of the car navigation apparatus 514 or 
providing information to the control program of the car navigation apparatus. 

Generation of the synthesized sound electrical signal 3 in the synthesizing 
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apparatus 4 is performed using the same method explained in the embodiments. Also, the 
hardware structures of the synthesizing apparatus 4 and extraction apparatus 10 may be 
the same as those explained in the embodiments above. 

Figure 33 is a block diagram to show an example of the structure of the apparatus 
in the receiving side (automobile side). In Figure 33, numeric symbol 507 represents a 
receiving apparatus, 508 a receiving antennae, 509 an amplifier (abbreviated as AMP in 
the diagram), 10 an extraction apparatus, 511 a speaker, 512 a synthesized sound, and 513 
a person (driver). 

Also, 515 represents an extraction timing switching apparatus, 516 an 
information storage apparatus, 517 a target district information database, 518 a route 
generation apparatus, 519 an input/output apparatus, 520 an extraction timing signal, and 
521a and 521b represent operation signals input by the driver. 

Overall flow of the signal will be explained with reference to Figure 33. 

The person (driver) 513 is driving while listening to radio broadcasting through a 
car radio in the vehicle. When information of interest is broadcast, by performing certain 
operations, an operation signal 521a is generated to inform the extraction timing switching 
apparatus 515. In response, the extraction timing switching apparatus 515 generates an 
extraction timing signal 520. 

Then, the extraction apparatus 10 starts the process of extracting information 502, 
and information 502 is automatically stored in the information storing apparatus 516. Then, 
later on, when the driver desires, by generating an operation signal 521b given by the 
driver's instruction, information stored in the information storing apparatus 516 is 
registered in the target district information database 517. The information thus registered 
in the target district information database 517 is able to be used as target district 
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information by the car navigation apparatus. 

The input/output apparatus 519 displays map data recorded on a medium such as 
CD (compact disc) and DVD (digital versatile disc) on a display apparatus (not shown), 
and, the route generation apparatus 518 generates pathways from the current location to 
the target destination location, and displays it on the display apparatus by superimposing 
on the map. 

In the following, some specific examples of providing commercial information 
using the present apparatus will be explained. In this example, as the original broadcast 
sound 501, a commercial message for a restaurant is heard. The commercial message 
conveys the following voice of an announcer along with the background music. 

[This is ABC restaurant located in Tokyo, Musashino city, Midoricho, telephone 
number is 0422-59-0000, Internet address is www.abc.co.jp]. Information 502 related to 
the original broadcast sound 501 includes the name of the restaurant, its longitude and 
latitude, telephone number, and URL (uniform resource locator), which are arranged in the 
text data sequence, for example, ABC restaurant, E135N30, 0422-59-0000, 
www.abc.co.jp". Such information 2 is superimposed on the original broadcast sound 1 
using the data hiding technique. 

Broadcasting station broadcasts such a synthetic sound as normal sound. In the 
meantime, the driver is listening to this broadcasting, and when the driver decides that it is 
of interest, reception button (not shown) is turned on and an operation signal 521a is 
generated. Also, an extraction timing signal 520 is generated by this operation. When the 
extraction apparatus 10 recognizes the operation signal 520, the information 502 is 
extracted as text data. 

The extracted information 502 is stored in the information storing apparatus 516, 
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and the driver can use this automatically registered restaurant information at any desired 
time. For example, the driver can phone the restaurant for reservation, and may set this 
restaurant as the destination in the car navigation apparatus. 

The route generation apparatus 518 in the car navigation apparatus generates 
route information on the basis of the longitude-latitude information contained in signal 2 
that was extracted earlier and the current longitude-latitude information, and guides the 
driver to the site by displaying the route on the map. 

Also, in the above embodiment, the timing for extraction operation for 
information 2 is when the driver listening to the broadcast decides that it is of interest and 
generates an operation signal 521a by operating a button, however, it is permissible not to 
adopt this approach, and, the structure may be arranged, for example, so that when radio 
broadcasting containing the information 2 begins, extraction of information 2 and storing 
action in the information storing apparatus 516 may be automatically started, and if it is 
found later that it is of no interest the driver can delete the registration. 

Also, in the explanation provided for the above embodiment, although a 
beneficial effect is obtained by inputting the information embedded in the voice signal 
broadcast in the car navigation apparatus of an automobile, resulting in increasing the 
utility of the car navigation apparatus, utilization method of the present invention is not 
limited to such an approach, and it is obvious that there are many other methods of 
utilization such as transmitting emergency information as information 2, or blinking the 
display lamp, or displaying words on the display apparatus. 

Also, in the explanation provided for the above embodiment, the synthesized 
sound electrical signal is transmitted by broadcasting, but it is clear that, instead of 
broadcasting to target a large number of listeners simultaneously, the invention can be 
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applied to transceivers to communicate on a one-on-one basis. 

Further, it is obvious that similar system may be applied not only for automotive 
devices but also to general household information devices. That is, by embedding various 
information necessary for operation of household electrical appliances in the sound source 
contained in television and radio broadcasting received by households, users are able to 
conveniently obtain information at low cost, enabling to operate household electrical 
appliance conveniently. 

Embodiment 7 

Next, Embodiment 7 of the present invention will be explained. Present 
embodiment enables to fuse broadcasting received in households and communication 
through the Internet. 

Presently, virtually every household has a radio receiver that can receive middle 
frequency broadcasting or short-wave broadcasting or FM broadcasting. Also, in recent 
years, increasing number of households have appropriate environment to enable 
connection to the Internet. 

Also, radio receiver is provided not only in households but also in automobiles in 
most cases to enable to listen to broadcasting. Also, there has been a rapid increase in the 
number of cars that are equipped with television receiver and car navigation system. 

Also, opportunity is increasing for using portable personal computer inside the 
automobile. 

Due to such recent widespread use of the Internet, program contents of 
conventional radio and TV, information related to programs and information to 
supplement the programs are often published through the Internet web page, and the 
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listeners/viewers are now able to listen/view broadcast programs with reference to such 
web pages. With this trend, URL of homepage is often broadcast during broadcasting. 
Also, URL notification is also given routinely by commercial broadcasts. 

However, when URL is publicized during programs and advertisements during 
radio and TV broadcasting, the listener/viewer interested in such information needs to take 
notes of such URL, and refer to the note to operate a keyboard to input the URL in PC, 
resulting in a cumbersome process. 

Also, it may also be considered to transmit text information such as URL through 
text multiplexing broadcasting, or so-called "visible radio", but this method presents a 
problem that it requires broadcast station to newly install expensive facilities and the 
receiver side must also have dedicated receiver. 

Further, information providing services attached to car navigation systems 
include Internet connection service, information portal service offered by car navigation 
system companies and providing information services based on conversation with 
operators; however, because contacts are made through public communication means, 
such as portable phones, in all cases, there are problems of cumbersome process and the 
need for service fees. 

The present embodiment enables to transmit voice sounds to users (receiving 
side) through broadcast waves as well as data other than the voice sounds, for example, to 
send URL data to be extracted at the receiving side, and, using a PC, rapidly access 
homepage of this URL on the Internet. 

Figures 34A, 34B are block diagrams of the structure of the present embodiment, 
and Figure 34A shows the sending side and Figure 34B shows the receiving side. 

In Figure 34A, numeric symbol 601 represents an original broadcasting sound 
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(audible sound signal), 602 a network address expressed in binary format different than 
the original broadcast sound 601, 3 a synthesized sound electrical signal, 4 a synthesizing 
apparatus, 605 a sending apparatus, 606 a sending antennae, and 607 a broadcasting 
station. Also, in Figure 34B, numeric symbol 608 represents a receiving antennae, 609 a 
receiving apparatus, 610 a speaker, 611 a sound collection apparatus (microphone), 612 a 
PC, 613 an extraction apparatus, 614 a control apparatus, 615 a NW access apparatus 
(browser), 616 a display apparatus, and 617 a network. 

Overall flow of the signal in Figures 34A, 34B will be explained. First, at the 
sending side (broadcasting station), using a data hiding technique for example, the 
broadcasting sound 601 and the network address 602 to be transmitted are synthesized 
electrically under conditions that a person is not able to detect the auditory presence of the 
network address 602, and the synthesized sound is input in the sending apparatus 605 as 
the synthesized sound electrical signal 3. In the sending apparatus 605, radio carrier wave 
is modulated with the synthesized sound electrical signal 3 and it is emitted from the 
sending antennae 606 as the sending electrical wave into the air space. 

At the receiving side, this sending electrical wave is captured by the receiving 
antennae 608, is demodulated in the receiving apparatus 609, and the synthesized sound 
electrical signal 3 is reproduced. Then, the synthesized sound electrical signal is input in 
the speaker 610, and converted to an acoustic signal, which is output. The person is not 
aware that the acoustic output contains network address 602 and listens to the sound as the 
original broadcast sound 1. 

The acoustic output is collected by the sound collection apparatus 611, is 
converted to the synthesized sound electrical signal and is input in a personal computer 
612. The personal computer 612, using the extraction apparatus 613, successively extracts 
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binary signals from the synthesized sound electrical signal, and reproduces the network 
address 602. 

This network address 602 is forwarded to the network access apparatus (browser) 
615 through the control apparatus 614, and the network access apparatus 615 accesses the 
network 617 and obtains information from the location corresponding to the network 
address 602 (homepage and the like). Then, the network access apparatus 615 displays the 
obtained information on a display apparatus 616 through the control apparatus 614. 

Figure 35 is a block diagram to show another example of the structure of the 
embodiment. In Figure 35, 618 represents a TV receiver, 620 the Internet, 621 a program 
sponsor, and 622 a viewer (PC user), and other numeric symbols are the same as those 
shown in Figures 34A, 34B. 

In Figure 35, at the broadcasting station 607, responding to request of the 
program sponsor 621, within the voice sounds of the TV program being broadcast or 
within the commercial message, the address of the homepage of the program sponsor 
established on the Internet using the sound-watermark technology. 

As a specific example, a case of an XYZ company promoting an xyz product will 
be explained. The XYZ Company provides original commercial sounds for the xyz 
product to the broadcasting station. Within the commercial message, the homepage 
address of the XYZ Company, for example, www.xyz.co.jp is embedded as a binary string, 
using the sound-watermark technology. 

The broadcasting station 607 broadcasts this synthesized sound in a manner 
similar to the images and normal commercial message. The viewer 622 viewing TV while 
operating the PC 612 listens to this commercial message and if the xyz product is of 
interest, when the viewer presses a certain key on the keyboard of the personal computer 



612, the synthesized sound flowing from TV is converted to the synthesized sound 
electrical signal through the microphone, and is input in the extraction apparatus of the 
personal computer. 

Then, the homepage address, www.xyz.co.jp, of the XYZ Company is extracted 
from the synthesized sound as a binary string. At this time, the extracted binary strings are 
successively accumulated, and each byte is decoded as a numeral or a word, and the 
network address (for example, the homepage address www.xyz.co.jp) is reproduced, and 
the page corresponding to this address is displayed by the browser on the display 
apparatus 616. 

Based on the homepage address, automatically or manually, the personal 
computer 612 accesses the homepage address on the Internet. Then, the user can order the 
product xyz through the EC (electronic commerce) site of the homepage. The user's ears 
hear the synthesized sound as normal sound, so that, to those users who are not interested 
in the service, the commercial message sounds normal. 

The frequency utilization band in present embodiment is the same as that 
explained using Figure 2. Also, the structure of the synthesizing apparatus may be the 
same as that explained in Figure 4. Also, the principle of operation of the synthesizing 
apparatus is the same as that explained using Figures 4A, 4B. Also, the structure of the 
extraction apparatus may be the same as that explained using Figure 5. Also, the 
operational procedures of the synthesizing apparatus and extraction apparatus is the same 
as those explained in Embodiment 1. 

It can be easily surmised from the explanations provided above relating to 
Embodiment 7 that the operation of the main steps related to the present invention can be 
realized by microprocessors in the personal computer executing relevant programs. 
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Especially, the receiver side in the embodiment is based on personal computers so that 
their main operations can be readily realized by a program executed by the computer. 

For such application programs, at the sending side, original broadcast sound 
(601) and a binary network address (602) different than the original sound are synthesized 
electrically to generate a synthesized sound electrical signal for modulation, and the 
modulated radio signal is received by the receiving apparatus, analyzed to demodulate the 
synthesized sound electrical signal, and generate an acoustic signal through the speaker, 
and this is collected by the sound collection apparatus to convert into the synthesized 
sound electrical signal, which becomes an input signal. 

In this program, a step of successively extracting the binary signals hidden in the 
synthesized sound electrical signal, a step of reproducing a network address based on the 
binary signals, a step of obtaining information by accessing the homepage on the Internet 
using the network address, and a step of outputting the obtained information are described 
as an executable program, and this program is stored in an executable form in the memory 
section of the personal computer. Then, this program is booted as necessary to perform the 
above steps. 

Such a program is pre-recorded in a computer-readable recording medium. 
Embodiment 8 

Embodiment 8 provides an additional function by fusion of the control method 
for a robot present in Embodiment 5 and broadcasting or communication. 

In other words, the purpose of the embodiment is not only to transmit voice 
directly to the robot, but also to operate the robot or to generate voices from the robot by 
using the aspects of TV broadcasting to transmit signals through the voice output of a TV 
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receiver. Further, an interactive system is to be realized between the viewer of TV and the 
broadcasting station to feedback through the robot to transmit the viewer's intentions to the 
station side. 

Figure 36 is a diagram to show an outline of the present embodiment, which 
shows an example of transmitting synthesized sound through TV broadcasting. In Figure 
36, numeric symbol 730 represents a TV receiver, 731 a TV screen, 732 a face of an actor 
on the TV screen, 733 a voice sound output from the TV receiver, 735 a robot, 736 a voice 
output from the robot, 737 an operation of the robot. 

Figure 37 is a block diagram of the functions of the robot shown in Figure 36, 
and in Figure 37, numeric symbol 741 represents a microphone, 742 an information 
extraction apparatus, 743 a motion control apparatus, 744 a robot operation command 
correlation table, 745 a driving mechanism, 746 a voice control apparatus, 747 a robot 
speech command correlation table, and 748 a speaker. 

In the following, the present embodiment will be explained in detail using 
Figures 36, 37. The voice sound 733 output from the TV receiver is the voice of the actor 
appearing on the TV show, and it is "How are you" in a natural language. The voice sound 
733 output from the TV receiver is, in fact, a synthesized sound, and within the natural 
language "How are you", a control signal for the robot is embedded in the actor's original 
voice (audible sound signal). 

And, , information regarding words and motion that appearing to respond to the 
voice sound 733 of the actor's words output from the TV receiver is embedded as the 
control signal in the audible sound signal to be transmit to the robot. In this example, the 
embedded control signals are word information to convey "I am fine" and motion 
information to "wave the right hand". 
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Figure 38 shows an example of the data for the robot operation command 
correlation table 744. And, Figure 39 shows an example of the data for the robot speech 
command correlation table 747. In other words, operations and speeches to correspond to 
such commands are pre-stored as a table in the robot. Here, "A" and "B" in Figure 38 are 
5 command codes for the robot, and operations to correspond to each command are defined 
in the table. Similarly, "0" and "1" in Figure 39 are also command codes for the robot, and 
speeches to correspond to the command codes are defined in the table. 

When it is desired to make the robot to vocalize "I am fine" and perform a motion 
to "wave the right hand", commands corresponding to "A" and "0" are embedded at the 
10 broadcasting station side in the voice sound "How are you" of the actor, and are broadcast. 
The receiving side receives this in the TV receiver 730, and outputs the voice sound of the 
actor "How are you". 

The robot 735 receives this voice output in the microphone 741, which is output 
as the synthesized sound electrical signal. The information extraction apparatus 742 
15 extracts a binary signal embedded in the synthesized sound electrical signal, and forwards 
it to the motion control apparatus 743 and the voice control apparatus 746. 

In the motion control apparatus 743, if "A" or "B" symbol is detected in the 
binary code, it searches in the robot operation command correlation table 744 according to 
the symbol to find the content of the operation (in this case, "wave the right hand"), and a 
20 signal corresponding to this operation is handed to the drive mechanism 745. The drive 
mechanism 745 carries out the operation of "wave the right hand" according to the signal. 

Also, in the voice control apparatus 746, if "0" or "1" symbol is detected from the 
extracted binary signal, it searches in the robot speech command correlation table 747 to 
find the content of the voice to be pronounced, and synthesizes the relevant voice and 
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outputs voice signal from the speaker. 

As another example, if it is desired to make the robot wave the left hand along 
with the speech of "I am fine", the control signal should be set to "B" and "0". 

In the above explanation, the robot speech command correlation table (747) is 
5 pre-registered in the robot and used, however, separate from this, without using such a 
correlation table, embedding the speed texts as part of the robot control signal may be 
considered. In the above example, ("A", "I am fine") may be used as the robot control 
signal. In this case, the robot extracts a text named "I am fine", and vocalize "I am fine" by 
voice synthesis. 

10 Figure 40 shows an example of realizing interactive TV broadcasting using the 

robot in the present embodiment. In Figure 40, numeric symbols 730, 731, 732, 735, 737 
as the same as those in Figure 36. Also, 734 represents voice output from the TV receiver, 
738 a voice output from the robot, 739a and 739b are touch sensors. 

Figure 41 is a block diagram to show the functional structure of the robot. In 

15 Figure 41, numeric symbols 741-748 are the same as those in Figure 37, and 749 

represents an input apparatus, 750 a control apparatus, and 751 a communication control 
apparatus. 

Figure 42 is a block diagram to show the relationship of broadcasting to network 
to realize a communication-broadcasting convergence system, and numeric symbol 752 
20 represents a robot, 753 a network, 754 a communication control apparatus, 755 a summing 
apparatus, 756 a display apparatus, 757 a player, 758 a broadcasting facility, and 759 a TV 
receiver. 

In the following, the present example will be explained with reference to Figures 
40-42. In this example, the voice sound broadcast on the TV is the voice of the actor 
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appearing on the TV, and represents a natural language that means "Those in favor, press 
the right hand, those opposing, press the left hand". A control signal correspond-ing to the 
words of the actor is embedded as the information 702 in the voice sound, and this 
information 702 is transmitted to the robot to produce desired motion and words. In this 
5 case, information 702 contains the word "which" and the operation information related to 
waving both hands. 

Figure 43 shows data for a robot operation command correlation table 744. Also, 
Figure 44 shows data for a robot speech command correlation table 747. The data in the 
correlation tables are pre-loaded in the robot. As shown in Figure 40, to let the robot 
10 pronounce the word "which" and to wave both hands, information 702 should be set to 
("A", "0"). 

As another example, if it is desired to let the robot to say "how are you" and to 
perform a welcome, information 702 should be set to ("B", "1"). 

In the example shown above, data in the robot speech command correlation table 
15 (747) is pre-registered in the robot, but, separate from such a method, speech texts may be 
embedded directly as information 702. When embedding speech texts, information 702 
may be made such that ("A", "DOCCHI"). In this case, the robot is made to extract a text 
"DOCCHI" and to pronounce "which" by voice synthesis. 

After which, the robot is readied to accept commands in the touch sensor 39a or 
20 39b provided in each hand. These touch sensors correspond to the input apparatus 749 in 
the case of the function block diagram in Figure 41. The viewer, according to the contents 
of programming, presses left hand touch sensor 39a or right hand touch sensor 39b. 

The input from the input apparatus 749 in Figure 41 is correlated to a part of the 
information output from the information extraction apparatus 742 and is sent to the 
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network 753 by way of the communication control apparatus 751. 

Here, information to discriminate such as network address (telephone number and 
the like), information to identify the broadcasting station, information to identify programs, 
and information to identify the words of the players are contained in a part of the 
5 information output from the information extraction apparatus 742. That is, it is the 

information to correlate the viewer input command to the exact portion of the broadcast 
program. 

As shown in Figure 42, information input and sent from a large number of users 
is collected in the collection apparatus 757 by way of the network 753, and the results are 
10 displayed on the display apparatus 756. The players 757 in the TV program and the person 
in charge view the numbers presented on the display apparatus 756 to carry out 
communication with the users. 

Embodiment 9 

15 Next, Embodiment 9 will be explained with reference to Figures 45-51. 

Embodiment 9 relates to the case of controlling a robot through a personal computer. 

Figure 45 is a diagram to show the overall structure of the robotic system, in 
which a personal computer (robot control signal generation apparatus) 801 and a robot 
shaped as an animal such as a bear 802 are connected by an audio line (electrical signal 

20 input means) 803. The robot 802 is configured such that the robot can nod using an 
internal servo motor (servo motor 826 in Figure 47), and can emit sounds into the air 
space from an internal speaker (speaker 822 in Figure 47). In this robotic system, the 
synthesized sound electrical signal and operation information are transmitted from 
personal computer 801 to robot 802 through the audio line 803. 



55 



Figure 46 is a function block diagram on the sending side, that is, the personal 
computer 801 side. The personal computer 801 is comprised by a robot control section 
(robot operation signal generation means) 811, a synthetic sound generation section 
(audible sound signal generation means) 812, a motor control section 813, an AM 
modulator 814, an adder (electrical signal generation means) 815, and a sound board 
(electrical signal generation means) 816. 

Figure 47 is a function block diagram on the receiving side, that is , the robot 802 
side. The robot 802 is comprised by an amplifying section 821, a speaker (emitting means) 
822, a bandpass filter (separation extraction means) 823, an AM demodulator 824, a motor 
drive section 825, and a servo motor (driving means) 826. 

The flow of signal will be explained with reference to Figures 46 and 47. First, in 
the personal computer 801, which is the sending side, the robot control section 811 
generates text information S801 and, from the generated text information S801, a motor 
rotation angle command value S802 for the servo motor 826, as a robot operation signal, is 
generated. As an example, it is also possible for the robot control section 811 to be 
constructed so that text information S801 and motor rotation angle command value for the 
servo motor 816 are generated from HTML (hypertext markup language) files, and 
electronic mail texts by performing morphological analysis and the like. 

Technology for generating text information S801 and motor rotation angle 
command value S802 using the morphological analysis technique will be explained 
briefly. 

First, morphological analysis is a technique for separating text files such as 
electronic mail into individual words (i.e., a morpheme) in order to provide a tag for each 
word, in order to analyze the contents of the text according to the morphological analysis 
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and to obtain tag information related to emotions. An actual case of morphological 
analysis will be explained using the following sentence. 

<Input mail text example> "Excuse me, this is Misae. This is an invitation to a home party 
at Sachan's house. This time, let's make it a pot-luck party." 
5 From these sentences, words to suggest a tag (abbreviated as cue-word) are 

extracted and emotion tag information is obtained. In the case of the above sentences, 
from words "excuse me" an emotion tag [greetings] is obtained, and from words "home 
party" an emotion tag [enjoyment] is obtained, and from the last words "pot-luck" an 
emotion tag [invitation] is obtained. 

10 Then, the cue-words for emotion tag information are stored in the speech 

database as voice sound files, that have been modulated with tonal parameters (pitch 
frequency, power, resonance) to provide linguistic tonality, so that when the input mail 
text is output, it is possible to output it as a speech pattern with some emotional content. 
Here, cue-words to be associated with emotion tag information may be 

15 considered to include the following. Cue-words for an emotion tag [happiness] may 

include "success" and "congratulations" and the like, cue-words for an emotion tag [anger] 
may include "complaints", "anger" and the like, cue-words for an emotion tag [sadness] 
may include "failure", "pain" and the like, cue-words for an emotion tag [pleasure] may 
include "laughter", "enjoyment" and the like, cue-words for an emotion tag [apology] may 

20 include "sorry", "forgive me" and the like, cue-words for an emotion tag [surprise] may 
include 'eh?', "whoa" and the like, cue-words for an emotion tag [emphasis] may include 
"!", "urgent" and the like. 

Then, in the present embodiment, to operate the robot in accordance with the 
emotion tag obtained by the morphological analysis, by correlating to the emotion tag 
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information, command values (motor number, location, speed, time) for the robot joint 
drive motor (servo motor 826 in the present embodiment) are arranged chronologically 
beforehand, and the results are stored in the gesture database. By so doing, operation 
signals for the robot are generated in reference to the jester database according to the 
emotion tag information obtained by using the morphological analysis. 

Figure 48 is a function block diagram to show the details of the robot control 
section 811 in the present embodiment. In the robot control section 811, the morphological 
analysis section 841 performs morphological analysis on the text information to extract 
cue-words, and the emotion tag information extraction section 842 obtains emotion tag 
information on the basis of the cue-words. The robot operation signal generation section 
843 generates robot operation signals on the basis of the emotion tag information obtain in 
reference to gesture database 844. 

Here, returning to Figure 46, the synthetic sound generation section 812 generates 
a synthesized voice sound S803 based on the text information S801 sent from the robot 
control section 811. Such a technology is known to be disclosed in, for example, in 
Nobuhiko Kitawaki editor, "Speech Communication Technology - Speech and Acoustics 
Technologies for Multimedia Services pp.62-86, Corona, 1996 (reference 5). 

Also, the motor control section 813 generates a motor control signal S804 from 
the motor rotation angle command value S802 sent from the robot control section 811. 
This motor control signal S804 is a value derived by a simple first-order equation from the 
motor rotation angle command value S802. Then, the motor control signal S804 is 
amplitude-modulated in the AM modulator 814 to generate an AM modulated wave S805. 
Here, the carrier wave for the AM modulated wave S805 may be selected in the vicinity of 
18 KHz so that the AM modulated wave would not be heard by human ears. 
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The AM modulated wave S805 and the synthesized sound signal S803 are 
synthesized in the adder 815, and are sent to the sound board 816 in the personal computer 
801, and are sent to the audio line 803 as a synthesized sound electrical signal S806. 

The synthesized sound electrical signal S806 sent from the personal computer 
801 passes through the audio line 803 and is input in the amplifying section 821 and the 
bandpass filter 823 of the robot 802, which is the receiving side. In the robot 802, the 
synthesized sound electrical signal S806 is amplified using the amplifier 821, and is 
emitted into the air space. Therefore, the person is able to hear the voice sound emitted 
from the speaker 822. 

In the meantime, the bandpass filter 823 having its center value at the AM carrier 
wavelength extracts the AM modulated wave from the synthesized sound electrical signal 
S806, and the AM modulator 824 reproduces the motor control signal S807 from the 
motor control signal S807. From the motor control signal S807, the motor drive section 
825 generates a servo motor command signal S808 for operating the servo motor 826. The 
servo motor 826, based on the servo motor command signal S808, rotates while 
controlling the rotation angle at the value preset by the sending side (personal computer 
801) and makes the robot 802 to oscillate its neck. 

Figure 49 shows an example of the relation, in the present embodiment, of the 
text information S801 from the sending side, synthesized sound signal S803, motor 
rotation angle command value S802, motor control signal S804, AM modulated wave 
S805, and the synthesized sound electrical signal S806. In this example, it is imagined that 
the robot 802 announces "mail from Kaori has arrived", and tilts its neck from left to right. 

Here, if it is desired for the robot to have a number of degrees of freedom, as 
shown in Figure 50, in the sending side (personal computer 801), a plurality of motor 
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control sections 813, AM modulators 814, and adders 815 are readied, and the AM 
frequency of the carrier wave should be varied slightly. Also, the receiving side should 
prepare in the same manner, and a plurality of bandpass filters 823, AM modulators 824, 
motor drive sections 825 and servo motors 826 should be readied. 

As explained above, according to the present embodiment of the robotic system, 
synthesized sound signal S803 and motor control signal S804 can be transmitted at the 
same time from the personal computer 801 to the robot 802 using only the audio line 803. 
Also, without providing a special structure, emission of sound from the speaker 822 and 
the neck oscillation operation of the robot 802 by the servo motor 826 can be carried out 
synchronously. Therefore, the robotic system can be simplified and the cost lowered. Also, 
because the signal line for operating the robot does not need to be dedicated in the 
personal computer 801, while the robot 802 is being operated, scanner, terminal adapter, 
printer and the like can also be operated simultaneously. 

Figure 51 shows a flowchart of the process of electrically synthesizing a sound 
signal. In the present embodiment, an application program for executing the steps in this 
flowchart is recorded on a hard disc (recording medium) of the personal computer 801. 
Such a program can also be recorded on other computer-readable recording medium such 
as floppy disc, opto-magnetic disc, ROM, CD-ROM and the like. 

First, in step S101, an audible sound signal from text information is generated, 
and in step S102, a robot operation signal is generated from the text information. Next, in 
step S103, a synthesized sound electrical signal is generated by synthesizing the audible 
sound signal generated in step S101 and the robot operation signal generated in step S102. 
Next, progressing to step S104, the synthesized sound electrical signal synthesize in step 
S103 is output. Here, in this flowchart, step S101 and step S102 are concurrently 
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processed, but step S101 may be carried out first followed by step S102. Conversely, step 
S102 may be carried out followed by step S101. 

Embodiment 10 

Embodiment 10 will be explained with reference to Figures 52 to 54. Here, in 
Embodiment 10, the parts that are the same as those in Embodiment 9 are given the same 
numeric symbols in Figures 52 to 54. 

The overall structure of the robotic system is the same as that in Embodiment 9, 
and is as shown in Figure 45. 

Figure 52 is a function block diagram of the sending side, which is the personal 
computer 801 side. The personal computer 801 is comprised by a robot control section 
(robot operation signal generation means) 811, a synthetic sound generation section 
(audible sound signal generation means) 812, a motor control section 813, an AM 
modulator 817, an adder (electrical signal generation means) 815, and a sound board 
(electrical signal generation means) 816. 

Figure 53 is a function block diagram of the receiving side, that is, the robot 802 
side. The robot 802 is comprised by an amplifier section 821, a speaker (emitting means) 
822, a bandpass filter (separation extraction means) 827, a frequency counter 828, a motor 
drive section 825, a servo motor (driving means) 826, and a timer circuit 829. 

The flow of signal will be explained with reference to Figures 52 and 53. First, in 
the personal computer 801, which is the sending side, the robot control section 811 
generates text information S801 and, from the generated text information S801, a motor 
rotation angle command value S802 for the servo motor 826, as a robot operation signal, is 
generated. It is permissible to adopt the method of generating the text information S801 
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and the motor rotation angle command value S802 as in Embodiment 9. 

The synthetic sound generation section 812 generates a synthesized voice sound 
S803 based on the text information S801 sent from the robot control section 811. It is 
permissible to adopt the same method of generating the synthesized voice sound S803 as 
5 that used in Embodiment 9. 

The motor control section 813 generates a motor control signal S804 from the 
motor rotation angle command value S802 sent from the robot control section 811. This 
motor control signal S804 is a value derived by a simple first-order equation from the 
motor rotation angle command value S802. Then, the motor control signal S804 is 
10 FM-modulated in the FM modulator 817 to generate an FM modulated wave S809. Here, 
the carrier wave for the FM modulated wave S809 may be selected in the vicinity of 18 
KHz so that the FM modulated wave S809 would not be heard by human ears. 

The FM modulated wave S809 and the synthesized sound signal S803 are 
synthesized in the adder 815, and are sent to the sound board 816 in the personal computer 
15 801, and are sent to the audio line 803 as a synthesized sound electrical signal S806. 

The synthesized sound electrical signal S806 sent from the personal computer 
801 passes through the audio line 803 and is input in the amplifying section 821 and the 
bandpass filter 823 of the robot 802 as the receiving side. In the robot 802, the synthesized 
sound electrical signal S806 is amplified using the amplifier 821, and is emitted into the 
20 air space. Therefore, the person is able to hear the voice sound emitted from the speaker 
822. 

In the meantime, the bandpass filter 827 having its center value at the AM carrier 
wavelength extracts the FM modulated wave from the synthesized sound electrical signal 
S806, and the frequency counter 828 measures the frequency of the FM modulates wave. 
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The motor driving section 825 generates a servo motor command signal S808 to drive the 
servo motor 826 according to the measured value of the frequency counter 828. The motor 
drive section 825 generates a servo motor command signal S808 for operating the servo 
motor 826. The servo motor 826, based on the servo motor command signal S808, rotates 
while controlling the rotation angle at the angle preset by the sending side (personal 
computer 801) and makes the robot 802 to oscillate its neck. Here, by the action of the 
timer circuit 829, the frequency measurement and servo motor command signal S808 are 
generated periodically. 

Figure 54, in Embodiment 10, is a diagram to show an example of the relation of 
sending side information text information S801, synthesized sound signal S803, motor 
rotation angle command value S802, FM modulated wave S809, synthesized sound 
electrical signal 806. In this example, it is imagined that the robot 802 announces "mail 
from Kaori has arrived", and tilts its neck from left to right. As can be understood from 
this diagram, in the present embodiment, the system is arranged so that the higher the 
motor rotation angle command value S802 the higher the frequency of the FM modulated 
wave S809. 

Here, if it is desired for the robot to have a number of degrees of freedom, as 
shown in Figure 50 in Embodiment 9, in the sending side (personal computer 801), a 
plurality of motor control sections 813, FM modulators 817, and adders 815 are readied, 
and the FM frequency of the carrier wave should be varied slightly. Also, the receiving 
side should prepare in the same manner, and a plurality of bandpass filters 827, frequency 
counters 828, motor drive sections 825 and servo motors 826 should be readied. 

According to the present embodiment of the robotic system, synthesized sound 
signal S803 and motor control signal S804 can be transmitted at the same time from the 
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personal computer 801 to the robot 802 using only the audio line 803. 

The same effects as those in Embodiment 9 can be obtained by the robotic system 
in Embodiment 10. That is, according to the robotic system in Embodiment 10, using only 
the audio line 803, synthesized sound signal S803 and motor control signal S804 can be 
5 transmitted at the same time from the personal computer 801 to the robot 802. Also, 
without providing a special structure, emission of sound from the speaker 822 and the 
neck oscillation of the robot 802 by the servo motor 826 can be carried out synchronously. 
Therefore, the robotic system can be simplified and the cost lowered. Also, because the 
signal line for operating the robot does not need to be dedicated in the personal computer 
10 801, while the robot 802 is being operated, scanner, terminal adapter, printer and the like 
can also be operated simultaneously. 

Also, in the robotic system in Embodiment 10, although the apparatus becomes 
somewhat more complex compared with the robotic system in Embodiment 9, compared 
-~ with the system using the AM modulator in Embodiment 9, more stable operation is 

; 15 possible and the acoustic effects are improved. 

Embodiment 11 

Next, Embodiment 11 will be explained with reference to the diagram in Figure 

55. 

20 Figure 55 is a system configuration diagram of a robotic system in Embodiment 

11, and difference between this embodiment and Embodiments 9 and 10 is that in 
Embodiments 9 and 10, the synthesized sound electrical signal synthesized by 
synthesizing the synthesized sound signal and robot operation signal is transmitted from 
the personal computer 801 to the robot 802 through the audio line 803, in Embodiment 11, 
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however, the synthesized sound electrical signal is transmitted through a radio 
transmission apparatus 804. 

In detail, the personal computer 801 is connected to the transmitter 804a of the 
radio transmission apparatus 804 through the audio line 803a, and the robot 802 is 
connected to the receiver 804b of the radio transmission apparatus 804, and the 
synthesized sound electrical signal generated in the personal computer 801 is transmitted 
to the robot 802 by way of the radio transmission apparatus 804. In the robotic system in 
Embodiment 11 also, the robot 802 amplifies the transmitted synthesized sound signal and 
emits it into the air space, and the robot operation signal is demodulated to oscillate the 
neck. Here, in Embodiment 11, the audio line 803b and the receiver 804b of the radio 
transmission apparatus 804 constitute the electrical signal input means of the robot 802. 

For the radio transmission apparatus 804, apparatus based on infrared or electrical 
waves may be considered. Also, in the robotic system in Embodiment 11, either the AM 
modulation method described in Embodiment 9 or the FM modulation method described 
in Embodiment 11 may be utilized. In the robotic system in Embodiment 11, the 
advantage is that physical transmission line is not necessary between the personal 
computer 801 and the robot 802. 

Embodiment 12 

Embodiment 12 will be explained with reference to the diagrams in Figures 56 

and 57. 

Figure 56 is a configurational diagram of the robotic system in Embodiment 12, 
and in Embodiment 12, there is no personal computer 801, a compact disc player 
(abbreviated as CD player below) 805 and a robot 802 are connected by an audio line 803. 
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Here, the robot 802 is the same as the robot 802 in Embodiment 9 or 10. 

And, in the robotic systems described in Embodiments 9, 10, the synthesized 
sound electrical signal is generated in the personal computer 801, but in Embodiment 12, 
the synthesized sound electrical signal is written in the compact disc (abbreviated as CD 
below) 806. Then, when this CD 806 is played in the CD player 805 to read the contents 
of CD 806, the synthesized sound electrical signal synthesized by the synthesized sound 
and the robot operation signal is sent from the CD player 805 to the robot 802 and the 
robot emits sounds from the speaker and oscillates its neck. 

Therefore, in this case also, it is possible to link the sound and motion of the 
robot 802. Also, in the robotic system in Embodiment 12, the advantage is that the 
personal computer 801 is not required. 

Figure 57, in the robotic system in Embodiment 12, is a diagram to show an 
example of the method of writing the synthesized sound electrical signal in a CD. The 
example of writing the synthesized sound electrical signal in the CD will be explained 
with reference to Figure 57. 

The structures of the robot control section 811, synthetic sound generation section 
812, motor control section 813, FM modulator 817, adder 815, are the same as the 
structures inside the personal computer 810 in Embodiment 10, and their explanations are 
omitted. The synthesized sound signal generated in the syntheses sound generation section 
812 and the FM modulated wave modulated by the FM modulator 817 serving as the robot 
operation signal FM are multiplexed in the adder 815 and becomes the synthesized sound 
electrical signal. This synthesized sound electrical signal is stored once in the disc 
apparatus 830. The stored data are quantized at 16-bits and are sampled at 44.1 KHz. 

Sampled data can be duplicated by pressing using a pressing apparatus 840 as in 
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normal musical CD to manufacture the CD containing the synthesized sound electrical 
signals. 

Embodiment 13 

Next, Embodiment 13 will be explained with reference to the diagrams in Figure 
58 and 59. 

The overall configuration of the robotic system in Embodiment 13 is the same as 
that in Embodiment 9, and will be explained using Figure 45. The robotic system is 
comprised by a personal computer (computer system) 801 and the robot 802, which are 
connected by the audio line (audio signal input means) 803. The robot 802 is able to 
oscillate the neck using the servo motor housed internally, and is able to emit sounds into 
the air space using the internal speaker. Here, the audio line 803 has an audio line L (left) 
channel 803L and an audio line R (right) channel 803R. 

Figure 58 is a function block diagram, on the sending side, that is, the personal 
computer 801 side. The personal computer 801 is comprised by a voice-music control 
section (audible sound generation means) 831, a motor control section 832, a modulator 
(audio signal generation means) 833, and a stereo audio output apparatus (audio signal 
outputting means) 834. The stereo audio outputting apparatus 834 has two channels, an 
L-channel and an R-channel, for outputting audio signals. 

Figure 59 is a function block diagram, on the receiving side, that is, the robot 802 
side. The robot 802 is comprised by a speaker (broadcasting means) 835, a demodulator 
(robot operation signal reproducing means) 836, a motor drive section 837, and a servo 
motor (drive means) 838. 

Next, the flow of signal will be explained with reference to Figures 58 and 59. 
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First, the personal computer 801, which is the sending side, the voice-music control 
section 831 generates an audible sound signal of voice or music, and outputs this audible 
sound signal to the stereo audio control section 834 as well as to the motor control section 
832. The motor control section 832 generates a motor rotation angle command signal from 
the audible sound signal input from the voice-music control section 831, and outputs it to 
the modulator 833. The modulator 833 modulates or encodes the motor rotation angle 
command signal input from the motor control section 832 to generate an audio signal, and 
output it to the stereo audio outputting apparatus 834. Here, the method of generating 
audio signal in the modulator 833 may include various methods such as AM modulation, 
FM modulation, PM modulation, PC modulation and the like. The carrier frequency of the 
modulated wave may be any frequency, but it is preferable to choose frequency regions 
that are inaudible to human ears. 

Then, the stereo audio outputting apparatus 834 sends the audible sound signal 
input from the voice-music control section 831 to the robot 802 from the audio signal 
outputting L-channel through the audio line L-channel 803L, and at the same time, sends 
the audio signal input from the modulator 833 for driving the motor to the robot 802 from 
the audio signal outputting R-channel through the audio line R-channel 803R. 

The robot 802, which is the receiving side, emits the audible sound signal input 
by way of the audio line L-channel 803 into the air space through the speaker 835. 
Therefore, the person is able to hear the sound or music emitted from the speaker 835. 

In the meantime, the audio signal input through the audio line R-channel 803R is 
input in the demodulator 836, and the demodulator 836 demodulates or decodes the audio 
signal to reproduce the motor rotation angle command signal, and output it to the motor 
drive section 837. The motor drive section 837 generates a servo motor command signal to 
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drive the servo motor, and outputs it to the servo motor 838. Thus, the servo motor 838 
rotates to the rotation angle preset by the sending side (personal computer 801) according 
to the servo motor command signal, and oscillates the neck of the robot 802. 

As explained above, according to the robotic system in Embodiment 13, it is 
5 possible to communicate between the personal computer 801 and the robot 802 by 

transmitting the audible sound signal through the audio line L-channel 803L and the robot 
operation signal through the audio line R-channel 803R, so that there is no need for a 
dedicated operation signal line. Therefore, the robotic system can be simplified and the 
cost lowered. Also, because the signal line for operating the robot does not need to be 

10 dedicated in the personal computer 801, while the robot 802 is being operated, scanner, 
terminal adapter, printer and the like can also be operated simultaneously. 

Also, in Embodiments 9-11, an audible sound signal and a robot operation signal 
are synthesized in the personal computer 801 to generate a synthesized sound electrical 
signal, and this synthesized sound electrical signal is transmitted to the robot 802 through 

15 the audio line 803 and others, and in robot 802, the robot operation signal is separated and 
extracted from the received synthesized sound electrical signal, and using the extracted 
robot operation signal the servo motor 826 is driven, so that it is necessary to provide 
synthesizing means for synthesizing the synthesized sound electrical signal and separating 
extracting means for separating and extracting the robot operation signal from the 

20 synthesized sound electrical signal, but in Embodiment 13, the audible sound signal and 
the robot operation signal are communicated independently so that the synthesizing means 
or separating extracting means become unnecessary, and the system is simplified. 

Embodiment 14 
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In Embodiment 14 to be explained next, the present invention is applied to a 
system for advertisement and sales promotion. 

Figure 60 is a block diagram to show the structure of the system in Embodiment 
14. This system, originated by a retailer's request or through a service offered by an 
advertising agent, embeds coupon information in a commercial message broadcast by 
voice only, or a commercial message broadcast by voice and image by a broadcasting 
station using the data hiding technique and the like, and the consumer receiving this 
message obtains an incentive such as discount from the retailer by accumulating points 
from the coupons. 

As shown in Figure 60, advertising agent has a synthesizing apparatus 904 that 
can embed digital information other than the voice sound in the voice sound, using the 
data hiding technique and the like. Also, this advertising agent is provided with a web 
server 921 and a consumer database 922. 

Also, the broadcasting station has a program sending apparatus 905 and a 
transmitter 906. 

Also, the consumer has a receiving antennae 908 for receiving broadcasts sent 
from the sending antennae 907, a receiver 909, an extraction apparatus 910 for extracting 
information embedded in the voice sound, and a personal computer 911. 

Also, the retailer has a terminal for receiving and viewing the information that is 
accumulated in the consumer database 922. 

Next, the flow of the signal in the present embodiment will be explained. The 
retailer produces a commercial message 901 for advertising own company, and request an 
advertising agent to embed coupon information in the commercial message 901, and 
broadcast this embedded commercial message. Upon receiving the request, the advertising 
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agent side produces coupon data 902 for the commercial message, and using a 
synthesizing apparatus 904, embeds a signal (digital signal) 2 based on coupon data 902 in 
the audible sound signal 1 in the commercial message 901, and outputs it as the 
synthesized sound electrical signal 3. Here, because the synthesizing apparatus 904 
synthesizes using the data hiding technique, a listener cannot recognize signal 2 in the 
synthesized sound output from the synthesizing apparatus 904. 

Coupon data 902 contains, at least, sending discrimination information for 
identifying dispatching of the commercial message, sending date and time, effective 
period of the coupon, and coupon point information. 

The synthesized sound electrical signal 3 is sent out according to broadcasting 
schedule by the program sending apparatus 905 in the broadcasting station, modulated in 
the transmitter 906, and is sent out from the sending antennae 907 as electrical waves. At 
the consumer side, based on the voice sound received in the receiving antennae 908 and 
demodulated in the receiver 909, signal 2 is extracted by the extraction apparatus 910, and 
the extracted signal is forwarded to the personal computer 911. The personal computer is 
provided with dedicated software, and the coupon data 902 is reproduced by executing 
this software, and the coupon points are accumulated. 

As the points are accumulated, the consumer can receive an incentive equal to the 
saved points by accessing the web server 921. In this case, communication between the 
personal computer 911 and the web server 921 is performed through the Internet, for 
example. The web server 921 receives the coupon data accumulated by this consumer 
from the consumer side, and after checking its legitimacy by matching with the original 
coupon data 902, writes the point data of this consumer in the consumer database. Here, 
because the web server 921 checks the sending discrimination information contained in 
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the coupon data 902, when the consumer side makes an request to accumulate points using 
a non-existent sending discrimination information or to accumulate points more than two 
times for the same sending discrimination information, such requests can be denied. 

Accordingly, information on the points written in the consumer database 922 and 
an effective period of a point can be inspected using a terminal 923 provided in the retailer, 
and based on this, the retailer awards the consumer a special incentive such as price 
discount. 

According to the present embodiment, because the consumer can not only enjoy 
the process of accumulating the points for obtaining incentives but the consumer also 
becomes actively interested in searching for broadcast commercial messages, so that the 
effect of advertising is increased, leading to enhanced sales. 

Here, the present embodiment relates to business transactions between retailer 
and consumer, but the present invention is not limited to this, it is possible to utilize the 
sales promotional system to all types of business transactions among any parties. 

The processes described above may be performed by recording application 
programs for performing the processes, and loading and executing the programs in a 
computer system to synthesize signals, transmit the synthesized signals, extract the signal 
from the synthesized sound, and to exercise various types of controls using the extracted 
signal. "Computer system", in this context, includes any OS (operating systems) and 
peripheral hardwares. 

Computer-readable recording media include portable media such as floppy disks, 
opto-magnetic disks, ROM, CD-ROM, as well as fixed memory devices such as hard 
disks housed in computer systems. Further, computer-readable recording media include 
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those that hold programs for a given length of time, such as volatile memories (RAM) 
contained internally in computers for servers and client computer systems used in 
transmitting application programs through networks such as the Internet or 
communication lines of telephone circuits. 

The above programs may be transmitted from the computer system storing such 
programs in a memory device and the like to other computer systems through a 
transmission medium or by transmission waves through the transmission medium. Here, a 
transmission medium for transmitting programs refer to a medium having a capability to 
transmit information such as networks (communication networks) represented by the 
Internet or communication lines represented by telephone circuits. 

Application programs may perform a part of the described functions. Further, 
they may be operated in conjunction with pre-recorded programs already stored in 
computer systems to provide differential files (differential programs). 

So far, various embodiments of the present invention have been described with 
reference to the diagrams, but the specific structures are not limited to those exemplified, 
and includes designs that can be included within a range of the essence of the present 
invention. 



73 



What is claimed is: 

1. A method for transmitting acoustic signals comprising: 

a synthesizing step for synthesizing an audible sound signal and an insertion 
signal to generate a synthesized sound electrical signal; 

an acoustic signal outputting step for converting the synthesized sound electrical 
signal to an acoustic signal and outputting the acoustic sound externally; 

a transmitting step for transmitting the synthesized sound electrical signal; and 

an extracting step for extracting said insertion signal from the synthesized sound 
electrical signal that has been transmitted. 

2. A method for transmitting acoustic signals according to claim 1, wherein said 
transmitting step relates to receiving an acoustic signal output in the transmitting step at 
the receiving side, which is again converted to the synthesized sound electrical signal . 

3. A method for transmitting acoustic signals according to claim 1, wherein said 
transmitting step relates to sending the synthesized sound electrical signal using a sending 
apparatus at the sending side, and the signal is received at the receiving side using a 
receiving apparatus. 

4. A method for transmitting acoustic signals according to claim 2, wherein, in said 
acoustic signal outputting step, a speaker is used as a means for outputting an acoustic 
signal, and, at the receiving side, a microphone is used as a means for converting the 
acoustic signal to the synthesized sound electrical signal. 
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5. A method for transmitting acoustic signals according to claim 1, wherein in said 
synthesizing step, as a method of synthesizing the audible sound signal and the insertion 
signal, a data hiding technique is used. 

6. A method for transmitting acoustic signals according to claim 3, wherein, in said 
transmitting step, said sending apparatus and said receiving apparatus perform sending 
and receiving operations through any one of methods including television broadcasting, 
radio broadcasting, cable television, police radio, public phone network, portable phone 
network, personal handy phone network, the Internet, and local area network. 

7. A method for transmitting acoustic signals according to claim 1, wherein 

said audible sound signal includes any one of a natural language sound, machine 
synthesized sound, music, warning sound, natural sound existing in natural world, sounds 
of animals and noises; and 

said insertion signal represents data expressed by a digital or analogue format and 
includes any one of a natural language sound, music, warning sound, noises, MIDI (music 
instrument digital interface) data, compressed acoustic data including MP3, multimedia 
signals including text, image, tactile signals, and olfactory signals; and provided is: 

a step of extracted signal processing for processing said insertion signal extracted 
in said extracting step. 

8. An apparatus for transmitting acoustic signals comprising: 

a synthesizing apparatus for electrically synthesizing an audible sound signal and 
an insertion signal, and outputting a synthesized sound electrical signal; 
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an acoustic signal outputting apparatus for converting the synthesized sound 
electrical signal and outputting the acoustic signal externally; and 

an extraction apparatus for extracting said insertion signal based on the 
synthesized sound electrical signal output from the synthesizing apparatus. 

9. An apparatus for transmitting acoustic signals according to claim 8, wherein a 
sound collection apparatus is provided to receive an acoustic signal output from the 
acoustic signal outputting apparatus and re-converting to a synthesized sound electrical 
signal; and the extraction apparatus extracts said insertion signal using the synthesized 
sound electrical signal converted by the sound collection apparatus. 

10. An apparatus for transmitting acoustic signals according to claim 8, wherein 
provided are: 

a sending apparatus for sending a synthesized sound electrical signal output from 
the synthesizing apparatus; and 

a receiving apparatus for receiving the synthesized sound electrical signal sent by 
the sending apparatus; wherein the extraction apparatus extracts said insertion signal using 
the synthesized sound electrical signal received by the receiving apparatus. 

11. An apparatus for transmitting acoustic signals according to claim 9, wherein said 
acoustic signal outputting apparatus is a speaker, and 

said sound collection apparatus is a microphone. 



An apparatus for transmitting acoustic signals according to claim 8, wherein said 
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synthesizing apparatus uses a data hiding technique as a means for synthesizing an audible 
sound signal and an insertion signal. 

13. An apparatus for transmitting acoustic signals according to claim 10, wherein 
said sending apparatus and said receiving apparatus performs sending and receiving 
operations through any one of methods including television broadcasting, radio 
broadcasting, cable television, police radio, public phone network, portable phone network, 
personal handy phone network, the Internet, and local area network. 

14. An apparatus for transmitting acoustic signals according to claim 8, wherein 
said audible sound signal includes any one of a natural language sound, machine 

synthesized sound, music, warning sound, natural sound existing in natural world, sounds 
of animals and noises; and 

said insertion signal includes data expressed by a digital or analogue expression 
and includes any one of a natural language sound, music, warning sound, noises, MIDI 
data, compressed acoustic data including MP3, multimedia signals including text, image, 
natural sounds existing in natural world, tactile signals, and olfactory signals; and 
provided is: 

an extraction apparatus for processing the signal extracted in an extracting step. 

15. An apparatus for transmitting acoustic signals according to claim 9, wherein 
said insertion is a machine control signal for controlling an operation of a 

machine; and provided are: 

a machine control section for controlling said machine according to the signal 
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extracted by the extraction apparatus; and 

a machine to be controlled for performing a pre-determined motion according to 
command of the machine control section. 

16. An apparatus for transmitting acoustic signals according to claim 10, wherein 
said insertion signal is a machine control signal for controlling an operation of a 

machine; and provided are: 

a machine control section for controlling said machine according to the signal 
extracted by the extraction apparatus; and 

a machine to be controlled for performing a pre-determined motion according to 
command of the machine control section. 

17. An apparatus for transmitting acoustic signals according to claim 9, wherein 
said insertion signal is a code information corresponding to a character string in a 

natural language; and provided are: 

a machine control section for controlling said machine according to the signal 
extracted by the extraction apparatus; and 

a sound generation machine for pronouncing the character string in a natural 
language corresponding to the code information extracted according to command of the 
machine control section. 



18. An apparatus for transmitting acoustic signals according to claim 10, wherein 

said insertion signal is a code information corresponding to a character string in a 
natural language; and provided are: 
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a machine control section for controlling said machine according to the signal 
extracted by the extraction apparatus; and 

a sound generation machine for pronouncing the character string in a natural 
language corresponding to the code information extracted according to command of the 
5 machine control section. 

19. A method for transmitting acoustic signals according to claim 3, wherein 
in said synthesizing step, an audible sound is divided into a plurality of channels 

and said audible signal in each channel and said insertion signal are synthesized 
individually to produce a synthesized sound electrical signal in each channel; 

in said acoustic sound outputting step, said synthesized sound electrical signal in 
each channel is converted to an acoustic signal which is output externally; 

in said transmitting step, said synthesized sound electrical signal is transmitted 
for each channel through a transmission path; and 

in a machine control step, the machine is controlled according to the signal 
extracted in the extracting step. 

20. A method for transmitting acoustic signals according to claim 19, wherein 
in a synthesizing step, each channel is controlled to determine whether to filter an 

20 audible sound signal only through an acoustic filter having a specific frequency band 
according to the insertion signal; and 

in an extracting step, the signal is extracted by analyzing the synthesized sound 
electrical signal transmitted according to a frequency property of the acoustic filter. 
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21 . A method for transmitting acoustic signals according to claim 1 9, wherein said 
channel is comprised by two channels represented by a left channel and a right channel for 
a stereophonic audible sound signal. 

22. A method for transmitting acoustic signals according to claim 21, wherein 

in said synthesizing step, an audible sound signal in each channel is input into a 
lowpass filter and a highpass filter, respectively, and, after controlling output levels of the 
highpass filter in both channels according to the insertion signal, each controlled output is 
synthesized with a filtered signal processed through the lowpass filter to generate a 
synthesized sound electrical signal for each channel; and 

in said extracting step, the synthesized sound electrical signal in each channel is 
individually input in the highpass filter, and the insertion signal is extracted by comparing 
output results from the highpass filters. 

23. An apparatus for transmitting acoustic signals according to claim 10, wherein 

a machine control section is provided to control an operation of a machine using 
the signal extracted by the extraction apparatus; 

the synthesizing apparatus is provided with a plurality of synthesizing sections 
for individually synthesizing an audible sound signal separated into a plurality of channels 
and the insertion signal; and 

the acoustic signal outputting apparatus is provided with a plurality of acoustic 
output sections for converting the synthesized sound electrical signal in each channel into 
an acoustic signal and exporting externally; and 

the sending apparatus and the receiving apparatus receive and transmit 



synthesized sound electrical signals in individual channels through transmission paths. 

24. An apparatus for transmitting acoustic signals according to claim 23, wherein 
the synthesizing apparatus is provided with an acoustic filter for transmitting only 

the audible sound signal having a specific frequency band according to an insertion signal; 
and 

the extraction apparatus extracts the insertion signal by analyzing the synthesized 
sound electrical signal transmitted according to a frequency property of the acoustic filter. 

25. An apparatus for transmitting acoustic signals according to claim 23, wherein 
said channel is comprised by two channels represented by a left channel and a right 
channel for a stereophonic audible sound signal. 

26. An apparatus for transmitting acoustic signals according to claim 25, wherein 
said synthesizing apparatus is provided with a lowpass filter and a highpass filter 

for inputting an audible sound signal of each channel; a level controller for controlling 
output levels of both channels of a highpass filter according to the insertion r signal; and a 
mixer for mixing a signal output from the level controller and a filtered signal processed 
through a lowpass filter; and 

said extraction apparatus is provided with a highpass filter for inputting the 
synthesized sound electrical signal of each channel and a comparator for extracting the 
insertion signal by comparing output results from highpass filters. 



An apparatus for transmitting acoustic signals according to claim 23, wherein 
said audible sound signal is a music signal; 
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said machine to be controlled is a robot; and 

said machine control signal is a dance operation signal for controlling a dance 
operation of the robot in synchronization with the music signal. 

28. An apparatus for transmitting acoustic signals according to claim 27, wherein 
said dance operation signal is a code signal to express a pattern of the dance 

operation of the robot. 

29. An apparatus for transmitting acoustic signals according to claim 27, wherein 
said synthesizing apparatus, when synthesizing a dance operation signal and an 

audible sound signal which is the music signal, allows said dance operation signal to 
precede said audible sound signal. 

30. A method for transmitting an acoustic signals comprising the steps of: 

a synthesizing step for synthesizing an audible sound signal and an insertion 
signal electrically to generate a synthesized sound electrical signal; 

a modulating step for generating a radio signal by modulating with the 
synthesized sound electrical signal; 

a transmitting step for transmitting the radio signal; 

a demodulating step for receiving the radio signal and demodulating the received 
signal to generate the audible sound signal; 

an acoustic signal outputting step for converting the synthesized sound electrical 
signal into an acoustic signal and outputting a converted signal; 

an extracting step for extracting the insertion signal according to the synthesized 
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sound electrical signal demodulated in said demodulating step; and 

a control step for controlling an apparatus according to the signal extracted in 
said extracting step. 

5 31. An apparatus for transmitting acoustic signals according to claim 30, wherein 
said demodulating step and said acoustic signal outputting step and said 
extracting step are performed in an apparatus installed in an automotive vehicle, and in 
said control step, a car navigation apparatus is controlled. 

32. An apparatus for transmitting acoustic signals according to claim 30, wherein 
included are: 

an information storing step for storing an information in a memory apparatus 
according to a signal extracted in said extracting step; and 

an information reading step for reading the information from the memory 
apparatus according to an operation performed by an operator. 

33. A method for transmitting acoustic signals according to claim 30, wherein 
said synthesizing step and said modulating step are performed by an apparatus 

provided in a broadcasting station; and 
20 said audible sound signal represents a broadcast content output by the 

broadcasting station. 

34. An apparatus for transmitting acoustic signals, comprising: 

a synthesizing apparatus for electrically synthesizing an audible sound signal and 
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an insertion signal to generate a synthesized sound electrical signal; 

a modulating apparatus for generating a radio signal modulated with the 
synthesized sound electrical signal; 

a demodulating apparatus for receiving the radio signal and demodulating to 
generate the synthesized sound electrical signal; 

an acoustic signal outputting apparatus for converting the synthesized sound 
electrical signal into an acoustic signal and outputting the acoustic signal; 

an extraction apparatus for extracting the insertion signal from the synthesized 
sound electrical signal demodulated by the demodulating apparatus; 

a control apparatus for controlling an apparatus by the insertion signal extracted 
by the extraction apparatus. 

35. An apparatus for transmitting acoustic signals according to claim 34, wherein 
said demodulating apparatus and said acoustic signal outputting apparatus and 

said extraction apparatus are installed in an automotive vehicle, and said control apparatus 
controls a car navigation apparatus. 

36. An apparatus for transmitting acoustic signals according to claim 34, wherein 
provided is a memory apparatus for storing information based on a signal extracted by the 

20 extraction apparatus and for reading the information stored according to an operation 
performed by an operator. 



37. 



An apparatus for transmitting acoustic signals according to claim 34, wherein 
said synthesizing apparatus and said modulating apparatus are provided in a 
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broadcasting station; and 

said audible sound signal represents a broadcast content output by the 
broadcasting station. 



5 38. An apparatus for transmitting acoustic signals according to claim 35, wherein 
said audible sound signal represents a broadcast content of a program or a 
commercial message; 

said insertion includes a name, a telephone number, location information, and 
information on a uniform resource locator (URL) that are related to the broadcast content; 
10 and 

said control apparatus controls so as to register names, telephone numbers, 
location information, and information on uniform resource locators (URL) that are related 
to contents of broadcasting in the car navigation apparatus. 

15 39. A method for transmitting acoustic signals according to claim 30, wherein 

said insertion signal represents a network address; and a network is accessed in 
said control step using a network address extracted in said extracting step, and information 
thus obtained is output. 



20 40. A method for transmitting acoustic signals according to claim 30, wherein said 
extracting step extracts said insertion signal according to a synthesized sound electrical 
signal converted from an acoustic signal output in said acoustic signal outputting step. 

41. A method for transmitting acoustic signals according to claim 40, wherein said 
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insertion signal represents a network address; and a network is accessed in said control 
step, using a network address extracted in said extracting step, and information thus 
obtained is output. 

42. A method for transmitting acoustic signals according to claim 30, wherein 
said audible sound signal represents a broadcast content of a program or a 

commercial message; 

said insertion signal represents a URL in a home page on the Internet; 
said transmitting step transmits the radio signal by broadcasting; and 
said control step controls so that a computer accesses the home page using the 

URL extracted to obtain the information. 

43. A method for transmitting acoustic signals according to claim 42, wherein 
said information obtained in said control step is information on a commercial 

product, and an ordering step is provided to enable the commercial product to be ordered 
through the home page. 

44. A method for transmitting acoustic signals according to claim 34, wherein 
said insertion signal represents a network address; 

said extraction apparatus extracts the network address; and 
said network accesses a network using the network address and an information 
thus obtained is output. 

45. An apparatus for transmitting acoustic signals according to claim 34, wherein 



86 

said audible sound signal represents a broadcast content of a program or 

a commercial message; 

said insertion signal represents a URL in a home page on the Internet; 

said radio signal generated by the modulation apparatus is a broadcast wave; and 

said control apparatus controls so that a computer accesses the home page using 

the URL extracted to obtain the information. 

46. A computer-readable recording medium containing a program, that receives as an 
input a synthesized sound electrical signal generated by receiving an acoustic signal input 
into a microphone, comprised by; 

an extracting step for separating and extracting an audible sound signal and an 
insertion signal; 

a regenerating step for regenerating a URL from said insertion signal; 
an information obtaining step for accessing a home page on the Internet using the 
URL and obtaining information from the home page of the URL. 

47. An apparatus for transmitting acoustic signals comprising; 

a synthesizing apparatus for electrically synthesizing an audible sound signal and 
an insertion signal to generate a synthesized sound electrical signal; 

a sending apparatus for generating a modulated radio signal by modulating a 
radio carrier wave with the synthesized sound electrical signal and sending the modulated 
radio signal; 

a receiving apparatus for receiving the modulated radio signal and demodulating 
the synthesized sound electrical signal according to the radio signal; 
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an acoustic signal outputting apparatus for converting the synthesized sound 
electrical signal into an acoustic signal and outputting the acoustic signal; 

an extraction apparatus for capturing the acoustic signal, and from a synthesized 
sound electrical signal thus obtained, extracting said insertion signal; and 

a control apparatus for controlling a robot according to said insertion signal 
extracted by the extracting section. 

48. An apparatus for transmitting acoustic signals according to claim 47, wherein 
said insertion signal extracted by the extraction apparatus represents a command 

to control an operation of the robot; and an operation command correlation table is 
provided that pre-defines relationships between commands and operations of the robot. 

49. An apparatus for transmitting acoustic signals according to claim 47, wherein 
said insertion signal extracted by the extraction apparatus represents a command 

to control an operation of the robot and sound generation; and provided are: a robot 
command operation correlation table that pre-defines commands and robot operations; and 

a robot speech command correlation table that pre-defines commands and robot 
sound generation. 

50. An apparatus for transmitting acoustic signals according to claim 47, wherein 
provided are: 

an input apparatus for a robot that is controlled by said control apparatus; and 
a receiving side communication control apparatus to send a signal input from the 
input apparatus to a network in order to feedback to the sending apparatus side. 



51. An apparatus for transmitting acoustic signals for controlling a robot, comprising: 
a synthesizing apparatus for generating a synthesized sound electrical signal by 

synthesizing an audible sound signal and a robot operation signal; 

a transmission path for transmitting the synthesized sound electrical signal; 

an extraction apparatus for receiving the synthesized sound electrical signal and 
extracting the robot operation signal from the synthesized sound electrical signal; 

a drive apparatus for driving a movable section of the robot using the robot 
operation signal extracted by the extraction apparatus; and 

an acoustic signal outputting apparatus for converting the synthesized sound 
electrical signal received from the transmission path and outputting externally. 

52. An apparatus for transmitting acoustic signals according to claim 51, wherein 
said synthesizing apparatus is provided internally in a computer equipped with a 

sound board, and the computer system outputs the synthesized sound electrical signal 
generated by the synthesizing apparatus to the transmission path by way of the sound 
board; and the transmission path transmits the acoustic signal by means of electrical 
signals or electromagnetic waves. 

53. An apparatus for transmitting acoustic signals, comprising: 

an audible sound signal generation apparatus for generating an audible sound 

signal; 

a robot operation signal generation apparatus for generating a robot operation 

signal; 
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a synthesizing apparatus for generating a synthesized sound electrical signal by 
electrically synthesizing the audible sound signal generated by the audible sound signal 
generation apparatus and the robot operation signal generated by the robot operation 
signal generation apparatus; and 

an outputting apparatus for outputting the synthesized sound electrical signal 
generated by the synthesizing apparatus. 

54. A method for transmitting acoustic signals comprising: 

a step of generating an audible sound signal on the basis of input text 
information; 

a step of generating a robot operation signal on the basis of the input text 
information; 

a step of synthesizing the audible sound signal and the robot operation signal; and 
a step of outputting the synthesized sound electrical signal. 

55. A computer-readable recording medium containing a program to execute steps 
comprising: 

a step of generating an audible sound signal on the basis of input text 
information; 

a step of generating a robot operation signal on the basis of the input text 
information; 

a step of synthesizing the audible sound signal and the robot operation signal; and 
a step of outputting the synthesized sound electrical signal. 
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56. An apparatus for transmitting acoustic signals comprising: 

an inputting apparatus for inputting a synthesized sound electrical signal 
generated by synthesizing an audible sound signal and a robot operation signal; 

an extraction apparatus for extracting the robot operation signal from the 
5 synthesized sound electrical signal input in the inputting apparatus; 

a drive apparatus for driving a movable section of the robot using the robot 
operation signal extracted by the extraction apparatus; and 

an acoustic signal outputting apparatus for converting the synthesized sound 
electrical signal input in the inputting apparatus into an acoustic signal, and outputting the 
10 acoustic signal. 

57. A method for transmitting acoustic signals comprising: 

a synthesizing step for synthesizing a synthesized sound electrical signal by 
processing an audible sound signal of a commercial message and coupon data relating to 
15 the commercial message; 

a sending step for sending the synthesized sound electrical signal by 
broadcasting; 

an accumulating step for receiving through a computer network the coupon data 
extracted from a broadcast made in said sending step, on the basis of the synthesized 
20 sound electrical signal, from a person receiving the broadcast, and accumulating the 
coupon data separately for each person receiving the broadcasting. 

58. A method for transmitting acoustic signals according to claim 57, wherein based 
on the coupon data accumulated in said accumulating step, a special incentive awarding 
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step is provided for awarding a special incentive associated with commercial dealings to 
the person receiving the commercial message. 

59. A method for transmitting acoustic signals according to claim 57, wherein 

5 said synthesizing step generates a synthesized sound electrical signal using a data 

hiding technique. 

60. A method for transmitting acoustic signals according to claim 57, wherein said 
coupon data include sending discrimination information to discriminate sending of 

10 broadcasts. 

61. A method for transmitting acoustic signals according to claim 59, wherein said 
accumulating step is related to checking of coupon data using the sending discrimination 
information. 

15 

62. A method for transmitting acoustic signals according to claim 58, wherein 
said coupon data include data relating point counts of the coupon; 

said accumulating step accumulates data related to counts of the coupon for each 
person receiving commercial messages through a computer network, and 
20 in said incentive awarding step, a incentive corresponding to the coupon counts is 

awarded to a person receiving the commercial message. 

63. A method for transmitting acoustic signals according to claim 57, wherein 
said computer network is the Internet. 
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64. A method for transmitting acoustic signals according to claim 58, wherein 

said synthesizing step and said accumulating step are performed by an advertising 
agent requested by an advertising owner of a commercial message. 
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ABSTRACT OF THE DISCLOSURE 
The acoustic signal transmission method is based on generating a synthesized sound 
electrical signal by electrically synthesizing an audible sound signal and another signal 
different than the audible sound signal at the sending side, and transmitting the 
synthesized sound electrical signal, and extracting the another signal different than the 
audible sound signal from the synthesized sound electrical signal at the receiving side. 
Here, generation of the synthesized sound electrical signal is made by using a data hiding 
technique, for example. Accordingly, the acoustic signal represented by the synthesized 
sound electrical signal can be heard by human ears in the same way as the audible sound 
signal, and the synthesized another signal cannot be detected by human ears. Here, the 
synthesized sound electrical signal can be transmitted as a sound wave in air space, as 
electrical signal through a transmission line or as radio signals such as infrared and 
electromagnetic waves. It is also possible to transport or distribute the synthesized sounds 
by recording on recording media such as compact disc and DVD. Also, using the signal 
extracted from the synthesized sound electrical signal, applications such as controlling 
machines such as robot, transmitting text data such as information for car navigation 
systems, computer network addresses and commercial business information. 
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COMBINED DECLARATION AND POWER OF ATTORNEY 
FOR UTILITY PATENT APPLICATION 



Attorney's Docket No. 
032590-077 



As a below-named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name; 

I BELIEVE I AM THE ORIGINAL, FIRST AND SOLE INVENTOR (if only one name is listed below) OR AN 
ORIGINAL, FIRST AND JOINT INVENTOR (if more than one name is listed below) OF THE SUBJECT MATTER 
WHICH IS CLAIMED AND FOR WHICH A PATENT IS SOUGHT ON THE INVENTION ENTITLED- 



ACOUSTIC SIGNAL TRANSMISSION METHOD AND ACOUSTIC SIGNAL TRANSMISSION APPARATUS 



the specification of which 

(check one) (3 is attached hereto; 

CD was filed on 



Application No. 

and was amended on 

(if applicable) 



I HAVE REVIEWED AND UNDERSTAND THE CONTENTS OF THE ABOVE-IDENTIFIED SPECIFICATION 
INCLUDING THE CLAIMS, AS AMENDED BY ANY AMENDMENT REFERRED TO ABOVE; 

I ACKNOWLEDGE THE DUTY TO DISCLOSE TO THE OFFICE ALL INFORMATION KNOWN TO ME TO BE 
MATERIAL TO PATENTABILITY AS DEFINED IN TITLE 37, CODE OF FEDERAL REGULATIONS, Sec. 1.56 
(as amended effective March 16, 1992); 

I do not know and do not believe the said invention was ever known or used in the United States of America before my 
or our invention thereof, or patented or described in any printed publication in any country before my or our invention 
thereof or more than one year prior to said application; that said invention was not in public use or on sale in the United 
States of America more than one year prior to said application; that said invention has not been patented or made the 
subject of an inventor's certificate issued before the date of said application in any country foreign to the United States 
of America on any application filed by me or my legal representatives or assigns more than twelve months prior to said 
application; 

I hereby claim foreign priority benefits under Title 35, United States Code Sec. 119 and/or Sec. 365 of any foreign 
applications) for patent or inventor's certificate as indicated below and have also identified below any foreign 
application for patent or inventor's certificate on this invention having a filing date before that of the application^) on 
which priority is claimed: 
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COMBINED DECLARATION AND POWER OF ATTORNEY Attorney s Docket No. 

032550-077 


COUNTRY/INTERNATIONAL 


APPLICATION NUMBER 


DATE OF FILING 

(day, month, year) 


PRIORITY 
CLAIMED 


Japan 


Patent 11-329914 


19/11/1999 


YES X NO 


Japan 


Patent 11-356827 


16/12/1999 


YES X NO 


Japan 


Patent 11-363811 


22/12/1999 


YES X NO 


Japan 


Patent 11-366345 


24/12/1999 


X 

YES NO 


Japan 


Patent 11-366346 


24/12/1999 


YES X NO 


Japan 


P2000-136716 


10/05/2000 


YES X NO 


Japan 


P2000-136717 


10/05/2000 


YES X NO 


Japan 


P2000-248800 


18/08/2000 


YES x NO 



I hereby appoint the following attorneys and agent(s) to prosecute said application and to transact all business in the Patent and 
Trademark Office connected therewith and to file, prosecute and to transact all business in connection with international 
applications directed to said invention: 



William L. Mathis 
i" Peter H. Smolka 
n Robert S. Swecker 
; ;Platon N; Mandros 
v- Benton S. Duffett, Jr. 
: Joseph R. Magnone 
: Norman H. Stepno 
=f Ronald L. Grudziecki 

Frederick G. Michaud, Jr. 

Alan E. Kopecki 
r!= Regis E. Slutter 

Samuel C. Miller, in 27,360 
j^Ralph L. Freeland, Jr. 



17,337 
15,913 
19,885 
22,124 
22,030 
24,239 
22,716 
24,970 
26,003 
25,813 
26,999 

16,110 



Robert G. Mukai 
George A. Hovanec, Jr. 
James A. LaBarre 
E. Joseph Gess 
R. Danny Huntington 
Eric H. Weisblatt 
James W. Peterson 
Teresa Stanek Rea 
Robert E. Krebs 
Robert M. Schulman 
William C. Rowland 
T. Gene Dillahunty 
Patrick C. Keane 



28,531 
28,223 
28,632 
28,510 
27,903 
30,505 
26,057 
30,427 
25,885 
31,196 
30,888 



Bruce J. Boggs, Jr. 
William H. Benz 
Peter K. Skiff 
Richard J. McGrath 
Matthew L. Schneider 
Michael G. Savage 
Gerald F. Swiss 
Michael J. Ure 
Charles F. Wieland m 
Bruce T. Wieder 
Todd R. Walters 



32,344 
25,952 
31,917 
29,195 
32,814 
32,596 
30,113 
33,089 
33,096 
33,815 
34,040 



Address all correspondence to: Robert E. Krebs, Esq. 

•'" Burns, Doane, Swecker & Mathis, L.L.P. 
P.O. Box 1404 

Alexandria, Virginia 22313-1404 

Address all telephone calls to: Robert E. Krebs. Esq. ; at (650) 854-7400. 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the 
United States Code and that such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 



FULL NAME OF SOLE OR FIRST INVENTOR 

Satoshi Iwaki 


SIGNATURE 

c SAM/ QroJs 


DATE x 

fi> /Ztf /oo 


RESIDENCE ^ 
429-2, Kamifuj isawa, Iruma-shi , Saitama-ken, Japan 


cnrzENSHip 

Japan 


POST OFFICE ADDRESS c /o NTT Intellectual Property Center, 9-11, Midori-cho 3-chome, 
Musashino-shi , Tokyo, Japan 


FULL NAME OF SECOND JOINT INVENTOR, IF ANY 

Akira Nakayama 


SIGNATURE f 


DATE /y 


RESIDENCE: ' ( 
35-10, -Namiki-cho, Hachioj i-shi , Tokyo, Japan 


J CITIZENSHIP 

J 


apan 


POST OFFICE ADDRESS c /o NTT Intellectual Property Center, 9-11, Midori-cho 3-chome, 
Musashino-shi, Tokyo, Japan 
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FULL NAME OF THIRD JOINT INVENTOR, IF ANY 

Tamotsu Machino 


SIGNATURE 


DATE /D / 

/D /00 


RESIDENCE 

7-5—206, Nishisumiyoshi , Tokorozawa-shi , Saitama— ken, Japa 


CITIZENSHIP 

i Japan 


POST OFFICE ADDRESS C / G NTT intellectual Property Center, 9-11, Midori-cho 3-chome, 
Musashino-shi , Tokyo, Japan 


FULL NAME OF FOURTH JOINT INVENTOR, IF ANY 

Ikuo Kitagishi 


SIGNATURE y 

Aim Kvca^UAJ^ 


DATE 


RESIDENCE 0 

534-1, Higashiasakawa-cho, Hachioj i-shi , Tokyo, Japan 


CITIZENSHIP 

Japan 


POST OFFICE ADDRESS c /o NTT Intellectual Property Center, 9-11, Midori-cho 3-chome, 
Musashino-shi, Tokyo, Japan 


FULL NAME OF FIFTH JOINT INVENTOR, IF ANY 

Yukihiro Nakamura 


SIGNATURE 


DATE 

iv/ x9 _ /oo 


RESIDENCE 

8-2-1, 1-1449-3, Sakuragaoka, Higashiyamato-shi , Tokyo, Jaj 


CITIZENSHIP 

>an Japan 


POST OFFICE ADDRESS c /o NTT Intellectual Property Center, 9-11, Midori-cho 3-chome, 
dlf Musashino-shi, Tokyo, Japan 


FULL NAME OF SIXTH JOINT INVENTOR, IF ANY 


SIGNATURE 


DATE 


RESIDENCE 

ru 


CITIZENSHIP 


POST OFFICE ADDRESS 


FULL NAME OF SEVENTH JOINT INVENTOR, IF ANY 


SIGNATURE 


DATE 


RESIDENCE 


CITIZENSHIP 


POST OFFICE ADDRESS 


FULL NAME OF EIGHTH JOINT INVENTOR, IF ANY 


SIGNATURE 


DATE 


RESIDENCE 


CITIZENSHIP 


POST OFFICE ADDRESS 


FULL NAME OF NINTH JOINT INVENTOR, IF ANY 


SIGNATURE 


DATE 


RESIDENCE 


CITIZENSHIP 


POST OFFICE ADDRESS 


FULL NAME OF TENTH JOINT INVENTOR, IF ANY 


SIGNATURE 


DATE 


RESIDENCE 


CITIZENSHIP 


POST OFFICE ADDRESS 


FULL NAME OF ELEVENTH JOINT INVENTOR, IF ANY 


SIGNATURE 


DATE 


RESIDENCE 


CITIZENSHIP 


POST OFFICE ADDRESS 
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